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ABSTRACT 


Determining the reliability of newly designed systems is one of the most important 
functions of the acquisition process in the military. Tracking the growth in reliability 
of a system as it is developed and modified repeatedly is an important part of the ac- 
guisition process. 

This thesis extends and expands a reliability growth simulation program written 
previously. It analvzes the capabilities and limitations of two discrete reliability growth 
models to determine which models are most applicable in estimating system reliability 
under a variety of different growth patterns. Negative growth patterns are also consid- 
ered. The result of this thesis is a FORTRAN simulation which enables a more accurate 
estimate of system reliability using test data generated during the development phase of 
an acquisition program. 


THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may not 
have been exercised for all cases of interest. While every effort has been made, within the 
time available, to ensure that the programs are free of computational and logic errors, 
they cannot be considered validated. Any application of these programs without addi- 


tional verification is at the risk of the user. 
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I. INTRODUCTION 


The reliability of a particular system or piece of equipment is one of the most im- 
portant factors in assessing its true value. A system with tremeridous capabilities that 
spends an inordinate amount of time in the maintenance facility is usually less desirable 
than an alternate system with relatively moderate capabilities that has a much higher 
availability. Therefore, accurately measuring the reliability of a proposed svstem during 
the test and design phases is an integral part of the materiel acquisition process. Tradi- 
tional reliability estimators rely upon conducting a sufficient number of tests or exper- 
iments so that a statistically valid point estimate or confidence interval can be 
established. These traditional estimates assume a constant or fixed reliability during anv 
particular design phase. Since reliability will almost certainly fluctuate from phase to 
phase of the acquisition process, methods are needed that track reliability as it changes. 
Additionallv, systems being produced presently are more complex and claim a much 
higher reliability than those produced just a few vears ago. Verifving high reliability in- 
herently requires significantly more testing than does verifving relatively moderate reli- 
ability since items are tested until failure. Therefore, all test data should be utilized to 
the utmost. 

The process of testing and evaluating newly designed pieces of eguipment is verv 
costly in terms of both time and money. Typically, the test engineer rnust face limitations 
in both of these resources and occasionally must compromise on either the extensiveness 
of the test design or the number of trials to be run or both. [t may be assumed that the 
constraints on time and money will not diminish in the near future -- if anything, these 
constraints will become tighter. Therefore, methods that are able to produce accurate 
estimates of svstem reliability for a smaller investment of resources are verv desirable. 
Reliability growth models are one such method. 

Reliability growth models make use of all available test data. Results from previous 
phases of design are combined with current test data so tliat a pattern of reliability 1s 
established. These models often make use of a procedure known as failure discounting. 
Failure discounting involves removing fractions of previous failures in order to make 
allowance for reliability improvement as the system evolves. The reliability pattern es- 
tablished by a reliability growth model becomes the basis for producing estimates of the 


actual system reliability. 


There are two types of reliability growth models -- discrete and continuous. A 
continuous model is based on the time until failure of the system under consideration. 
Obviously, a number of distributional assumptions are necessary in a model of this type. 
Discrete reliability growth models are employed when test data references the number 
of system tests and failures in a particular design phase. In order to use a discrete reli- 
ability growth model one must be able to classify a test result as either a failure or a 
success. In other words, the test data must be attribute data. In this thesis, situations 
where test parameters may classify a trial as a partial success or a partial failure are not 
allowed. 

This thesis addresses discrete reliability growth models. The major objective is to 
analyze the capabilities and limitations of two such models. Two different methods of 
discounting previous failures are also evaluated. A previously designed simulation pro- 
gram [Ref. 1: pp. 36-46] is expanded or modified as necessary in order to more fully de- 
velop the two discrete reliability growth models being examined. The original simulation 
uses actual reliabilities and Monte Carlo techniques in order to generate a random reli- 
ability growth pattern. The test data generated bv this growth pattern is then used in the 
reliability growth models in order to produce estimates of system reliability. A variety 
of different reliability growth patterns and a broad spectrum of failure discounting pa- 
rameters are systematically evaluated. At each phase, the estimate of system reliability 
is compared to the actual reliability. The applicability of the two discrete reliability 
growth models, to include limitations and capabilities is also addressed. 

The following chapter describes each of the two discrete reliability growth models 
being considered. An explanation of the two failure discounting methods is also offered. 
Additionally, work done in this area previous to this analysis is summarized. Chapter III 
discusses the methodology used io conduct the analysis, to include the computer simu- 
lation. Chapter III also describes the various different growth patterns that were ana- 
Iyzed. Chapter IV presents the results of the analysis. Chapter V details conclusions and 


recommendations. 


D 


II. DISCOUNTING PROCEDURES AND DISCRETE GROWTH 
MODELS 


A. BACKGROUND . 

In this chapter, two discrete reliability growth models are described. Two separate 
techniques of discounting failures are also explained. Both methods remove a fraction 
of a previous failure when the system successfully completes a trial without a reoccur- 
rence of the particular failure cause. Reliability is defined as the probability of the sys- 
tem successfully completing a single trial. The exact definition of a test trial is much 
more difficult to fix since, tvpically, the constitution of a trial is the responsibility of the 
testing agency. Normallv, a trial may be considered as the exercising of the particular 
system in a manner consistent with its purpose. If a weapons system is being evaluated, 
for example, then a trial could consist merely of a single attempt at target acguisition 
or proceed through target acquisition to an attempt at target destruction. The precise 
definition of a trial depends in large measure on the purpose or goal of the test. For the 
purpose of this thesis and in order to utilize discrete reliabilitv growth models, a trial 
must be defined so that it is discrete and can be evaluated as either a success or failure. 
Partial successes are not allowed, 

A testing phase may be defined as a number of trials, one or more, during which 
the configuration of the system is unchanged. Therefore, during a testing phase the ac- 
tual system reliability remains unchanged. A test phase mav consist of one or more 
system failures. If improvements or changes to the system are effected after each system 
failure (a test-fix-test scenario) then a phase would consist of one system failure and all 
of the successful trials leading up to that failure. Alternatively, the test design may dic- 
tate that testing be continued until a certain number of failures occur. Under this sce- 
nario (test-find-test) the cause of system failure is identified but system configuration 
remains unchanged until the pre-designated failure occurs. In general, a phase will en- 


compass all of the trials between system configuration changes. 


B. FAILURE DISCOUNTING 
Testing conducted during the initial design stages of a particular system often indi- 
cates low reliability. Generally, weaknesses in the configuration of the system or defects 


in the quality of its components cause system failure. Test designs are established so 


that the cause for these failures can be identified and corrected. Theoretically, then, as 
a weakness or a defect is identified and, hopefully, corrected the probability of that 
particular weakness or defect reoccurring should be reduced. This reduction in the 
probability of occurrence of a certain failure cause leads to improved system reliability. 
This concept is fully utilized in failure discounting. 

In order to effectively discount previous failures it is critical that the cause of the 
failure be properly identified. The level of detail that one wishes to ascribe to this iden- 
tification process is dependent upon the type of system being evaluated and the purpose 
of the test. If a complex system is being evaluated then a failure cause may be failure 
of a certain component or sub-component. The precise element that caused system 
failure is not critical but the ability to assign a failure cause to each system failure 15. 
Correctly determining failure cause is very difficult, particularly when de:ling with com- 
plex systems. Therefore, it is conceivable that design changes do not improve svstem 
reliability -- in fact, these changes may even degrade reliability. To apply the failure 
discounting procedures described below, one must be able to assign a failure cause to 
every system failure even though it may be done erroneously. 

1. Standard or Straight Percent Failure Discounting 

The standard discounting method will reduce a previous failure by a fixed 
amount after a predesignated number of trials have been successfully conducted without 
a reoccurrence of a failure due to the same cause. This method has been detailed fully 
in previous work [Ref. l: pp. 14-17] so only a brief description and an example of its 
application will be included here. 

To employ the standard discounting method one must specify two parameters 
- the fraction of a failure to be removed, F, and the discount interval or the number of 
successful trials, I, that must occur before application of the discount. For the purpose 
of this paper and to use the simulation described herein, these parameters are specified 
at the outset and remain constant throughout the test. The number of successful trials 
since the occurrence of the failure is referred to as T. These three values, F, I, and T 
are then used to discount previous failures at the end of each phase. 


Mathematically, the standard discounting method is given by Eguation 2.l: 
o 
DISCOUNTED FAILURE = (1 — EJ") (2.1) 


The term INT in the above equation refers to the integer portion of the argument 


T i s 
Cr . As such, if the number of successful trials, since the failure cause last occurred, 


T, is less than the number of trials required bv the discount interval, I, then INT( —) 
will be zero thus causing no discounting to be applied. Additionally, should a failure 
cause reoccur then the value of T is returned to zero thus removing any previous dis- 
counting that may have been applied. This ability, in effect, acts as a penalty factor. 
If a failure cause should reoccur then previous design changes may not have been ef- 
fective and thus any prior failure discounting could be unwarranted. The rate and 
amount of failure discounting can be controlled by altering F and I. If the system under 
evaluation is relatively simple and a proven failure identification process is in place, then 
a large discount to svstem failures may be warranted. This will result in a prediction of 


rapid reliability growth. 


Table 1. STANDARD DISCOUNT METHOD EXAMPLE 
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A small example of the application of the standard discount method will aid in 
clarifving its abilities. Consider the data in Table 1. This test data is purely fictitious 
and is intended solelv to illustrate the use of the standard discount method. We will 
assume that improvements or repairs are effected after each failure so actual system re- 
hability is not constant. 

Table 1 represents three phases of a test-fix-test scenario where two failure 


causes, X and Y, have been identified. It should be noted that under a test-fix-test plan, 


a phase will terminate with system failure. The result of each trial is listed as either a 
success or failure for a particular failure cause. 

The two parameters necessarv to employ the standard discount method are the 
fraction of failure to be removed, F, and the discount interval, I. Assume that F = 0.50 
and I = 3. System failure number two, attributed to failure cause Y, terminates phase 
two. At that point the failure due to cause X that terminated the first phase has had 


three successive successful trials so failure discounting can be applied: 


3 
ADJUSTED FAILURE = (1 — 0.50) 3) = 0.50 


Thus, for reliability computations at the conclusion of the second phase, the failure that 
occurred during phase 1 is only counted as one-half of a failure. 

Phase three terminates with a failure due to cause X. The value for T then 1s 
set to zero and thus when applied to the failure that occurred in phase one, full value 1s 
restored. At the end of phase two this failure had been discounted to 0.50 but, because 
the fallure cause reoccurred, this discount is viewed as unwarranted and the failure 1s 
returned to one. However, at the end of phase three failure cause Y has had seven suc- 


cessful trials. Therefore, discounting is applied: 


Aa INNE) _ 3 
ADJUSTED FAILURE = (1 — 0.50) 3 = (0.50) = 0.25 


Therefore, when computing reliability estimates at the end of phase three, only 2.25 
system failures (failure number one plus failure number three plus the discounted value 
of failure number two) are considered instead of the three failures that actually occurred. 

The standard discount method described is very flexible because of its two pa- 
rameters. Setting values for F and I does, however, require a good deal of professional 
Judgement and expertise. To avoid conflicts in setting such values David K. Lloyd pro- 
posed an alternate method of discounting failures [Ref. 2]. The Lloyd method is de- 
scribed in the next section. 

2. Lloyd Failure Discounting 

Discounting previous failures using the Lloyd discounting method is based on 
the premise that there should be some sort of statistical basis for determining how much 
a previous failure should be discounted. Lloyd offers the upper confidence limit of the 


probability that a failure cause will occur again as the discounted value of its respective 


fallure. The confidence limits for this probability are based on the number of successful 
trials since the failure cause last occurred. Under a test-fix-test scenario this is equivalent 
to the number of successful trials since the last failure. This number was defined in the 
description of the standard discount method as T. In order to implement Llovd failure 
discounting, one must set the level of confidence desired for the confidence bound. Since 
the method 1s applied on every trial there is no requirement to specify an interval, al- 
though as will be explained in a later chapter, specifying an interval for the Llovd 
method can be done and may be advantageous in some situations. 


The Lloyd discount method may be mathematically expressed as follows: 
ADJUSTED FAILURE =1—(1— CDT fT>0 (2:2) 


In the above equation CI is defined as the level of confidence desired and T is the 
number ot successful trials. If T has a value of zero then the value of the adjusted failure 
Is set to one. Setting the value of the adjusted failure equal to one when T equals zero 
gives the Llovd method the ability to restore all of the value of a discounted failure 


should its failure cause reoccur. 


Table 2. LLOYD DISCOUNT METHOD EXAMPLE 
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Table 2 illustrates the use of the Lloyd method. The test data 1s the same data 
used to demonstrate the standard discount method. Since the Lloyd method is applied 
on everv trial a fourth column has been added to the table reflecting the current value 
of the adjusted failure. Let us assume that the confidence interval desired in this case 
Was 5 

If reliability is computed at the completion of the third phase then 2.28 system 
failures would be used under the Llovd discounting method as opposed to the actual 
value of three that occurred. The last column of the table indicates the speed that failures 
become discounted with the Llovd method. 

The two discrete reliability growth models that will be examined are described 
in the next section. Each model makes use of discounted failure data and each has 
demonstrated, through simulation, the ability to estimate constant and concavely in- 


creasing reliability with reasonable accuracy. 


C. DISCRETE RELIABILITY GROWTH MODELS 

Both of the discrete reliability growth models to be presented assume a constant 
svstem reliability within a phase. The first model to be examined will be the Maximum 
Likelihood Estimate with Failure Discounting (MLEFD). This model is merely a deriv- 
ative of the conventional single phase maximum likelihood estimate. The single phase 
maximum likelihood estimate considers only test data generated during the current 
phase. The MLEFD, conversely, is cumulative in that it considers all of the available 
test data. The test data from early phases during which the actual system reliability was 
different requires adjustment. Thus, the use of failure discounting techniques provides 
a method by which failures occurring early in the development process can be assim- 
ilated into current reliability estimates. The second model is a regression model based 
on an exponential single phase reliability estimate. It estimates reliability for each phase 
using an exponential model and then performs a linear regression on these estimates to 
obtain a current estimate. The simulation program that was written to evaluate these 
two models was designed so that failure discounting may or may not be invoked with 
either model. 

Before offering a general description of the two models it is important at this 
juncture to digress for a moment and discuss the exact methodology that will be used to 
incorporate discounted failures into the various reliability estimates. As was seen in the 
section prior, discounted or adjusted failures are rarely if ever integer-valued. While this 


does not present a problem for the MLEFD reliability growth model it does pose an 


obstacle to using the exponential regression model. The convention devised in Captain 
Drake s thesis [Ref. 1: p. 23] will be continued in this paper. The number of trials up to 
and including a particular failure will be divided by the adjusted failure so that an ad- 
justed number of trials is computed. The adjusted failure value is then returned to one 


from its previous fractional value. The formula for computing adjusted trials is then: 


USTED TRIALS = DOS E (2.3) 
The appeal of this method is that the result is an integer number of failures while main- 
taining a constant ratio of number of failures to number of trials. While the adjusted 
number of trials may not be integer-valued they may be adjusted to integers, if necessary, 
through rounding with a much smaller effect on reliability computations than rounding 
adjusted failures. 

If a failure had been discounted (using either discounting method) from one to .25, 
for example, and the number of successful trials since the failure occurred was ten then 
the ratio of failures to successes would be .25 to ten. Using the method described above, 
the adjusted failure and actual trials are divided by the adjusted failure vielding a failure 
to success ratio of one to forty. Thus, the number of trials has been adjusted from ten 
to forty and the number of failures has been returned to one from .25. Since the ratio 
of failures to successes remains the same reliability computations will not be affected. 
Both of the models described below will make use of this method in computing reliability 
estimates. 

1. Maximum Likelihood Estimate with Failure Discounting 

The traditional estimate of system reliability is the maximum likelihood esti- 
mate. Letting R denote the estimate of reliability, the maximum likelihood estimate may 


be expressed as follows: 


pl TOTAL TRIALS — TOTAL FAILURES 
TOTAL TRIALS 


or alternatively: 


O CESSFUL TRIALS 


de TOTAL TRIALS 


As stated previously, the maximum likelihood estimate assumes a constant re- 


liability between phases. It also requires a large number of trials to accurately estimate 


the underlying system reliability. Failure discounting is used to make previous test data 
compatible with current results and thus increase the total number of trials available. 
However, since the actual system reliability is unknown it becomes a very non-trivial 
task to select the appropriate discounting method and the correct parameters. 

When previous test data is made available through the use of a discounting 


routine the equation for estimating the system reliability becomes: 


o TOTAL TRIALS 


or if the convention of restoring the number of failures to integer values is adopted 


TOTAL ADJUSTED TRIALS — TOTAL FAILURES 
TOTAL ADS D5 


R= (2.4) 

To utilize the MLEFD reliability growth model one first records all of the test 
data from the current phase. The discounting method to be used is then performed on 
the previous test data and an adjusted failure value is computed for each previous failure. 
The adjusted number of trials for each failure is then computed using Equation 2.3. The 
estimate of reliability for the current phase is then calculated using Equation 2.4 above. 

One deficiency of the maximum likelihood estimate is that, when testing 1s ter- 
minated after a fixed number of failures, the MLE is inherently a biased estimator of the 
actual reliability. The expected value of the maximum likelihood estimate has been de- 


rived in previous work [Ref. 3: p.34]. It may be expressed mathematically as follows: 
e (<) nu =P) 2 (2.5) 


where R represents the actual reliability of the system. The bias, B(R), in the estimate 
may be calculated using the formula: 


B(R) = E[R]—R 


Table 3 below depicts the performance of the maximum likelihood estimate for 
different values of actual reliability, R. The bias is inherent to the estimate in that it will 
be present regardless of the validity of the assumption of constant phase reliability. 

The use of failure discounting can offset the effect of bias as well as handle 
changing system reliability. However, this puts a lot of additional weight on the ability 


to correctly select the best method and choose the proper parameters. Alternatively, if 


an estimator of system reliability had to be biased one would want it to be conservatively 
biased as is the MILE. Underestimating actual system reliability is normally less costly 


than overestimating. However, a less biased estiniate would be beneficial. 


Table 3. INHERENT BIAS IN THE MAXIMUM LIKELIHOOD ESTIMATE 
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2. Exponential Regression Estimate 
The exponential regression reliabilitv growth model was developed by H. 
Chernoff and W.M. Woods and based on the exponential single phase estimate [Ref. 
4]. The derivation of this model has been developed fully in previous work [Ref. 5: pp. 
3-5] so for the purposes of this paper a general summary of results will be offered. 
The basic form of the exponential single phase estimate for a phase k 1.e., after 


the kth improvement or alteration has been made, is stated in Equation 2.6: 
R, = 1-02" (2.6) 


where R, is the reliability estimate and the coefficient of the exponential term, Y, can 


be calculated as follows: 


7, => a aa 


Zb totu, op fuel 


where X, is the number of trials up to and including the first failure in phase k. If X, 
equals one then Y, is defined equal to zero. It should be noted that this estimate was 
developed for the special case of testing until the first failure is encountered. If system 


failure occurs on the first trial (X, = 1) then Y, will equal zero and system reliability will 


be estimated as 1 — e? = 1-1 = 0. If failure occurs on the third trial, then the expo- 
nential single phase estimate would yield | — e- 1+7) — 0.777. 

[n order to evaluate the properties of the exponential single phase estimate a 
small simulation was designed. Geometric test data with actual reliability R were gener- 
ated using a transformation of a standard Uniform(0,1) random number. Ten thousand 
sets of geometric data were generated for each value of R. Table 4 represents the results 
of this simulation. Also included in the table is a simulation of the maximum likelihood 
estimate, (R), using the same test data. 

The values generated for the mean of the maximum likelihood estimate, R 
conform fairly well with the theoretical results, thus the simulation itself 1s accurate. The 
values generated for the exponential estimate, R , while still underestimating the actual 
reliability, are clearly less conservatively biased than the maximum likelihood estimates. 
The sample variance and mean square error of each estimate is also represented in the 
table. The mean square error of each estimate is the sum of the variance and the square 
of the bias. It can be seen that as the actual reliability exceeded .70 the mean square 
error for the exponential estimate became less than the mean square error of the maxi- 
mum likelihood estimate. Therefore, the advantage of a lower variance for the maximum 


likelihood estimate at values of actual reliability below .90 is offset by the greater bias. 


Table 4. PROPERTIES OF THE EXPONENTIAL RELIABILITY ESTIMATE 


KIEJ Maximum Likelihood Estimate DE Estimate 
ELK] 


è [rad [asai] R | vať) | useň 
OSO [0307] 0302 | 0101 | ori | 0387 | 0137 | 0157 — 
[oo [07 | 0.73 | 00157 | 0403 | 0797 | 0080 | 00% 
oss [osa | osa | 0050 [ 0062 | ossi | 0047 | 005 
















12 


The exponential regression reliability growth model developed bv ChernolT and 
Woods uses the technique of linear regression to estimate the coetlicient of the expo- 


nential term. The model can be expressed mathematically as follows: 


o = 


oe tbe "TAA (2.7) 


In the equation above k denotes the testing phase being used to compute the 
system reliability. x and D are computed at the end of each phase using the techniques 
of linear regression thus vielding an estimate of svstem reliabilitv. The exact derivation 
of the linear regression formula for x and f is detailed in other sources [Ref. 3: p. 3] and 
is provided at Appendix A for the interested reader. 

This model requires that both failures and trials be integers. Therefore, wlien 
adjusted trials are computed using Equation 2.3, the value obtained must be rounded to 
the nearest integer. Since linear regression is emploved, this model has the ability to 


track changing reliability without the use of failure discountins. 


D. SUMMARY OF PREVIOUS WORK 

Analysis of reliability growth models has been a continuing process. and many 
advances have been made in determing their properties [Refs. 6,7]. It would require many 
pages to summarize all of the previous work in this area. For the purposes ot this paper. 
the summarv will detail only those results which have a direct bearing on the simulation 
being used to evaluate the two reliabilitv growth models. 

A simulation to evaluate these models was constructed by Captain James Drake 
[Ref. 1: pp. 36-46]. This simulation was written to handle growth patterns where actual 
reliability was constant or growing at a fixed rate. Since one of the objectives of this 
paper is to expand the number of reliabilitv growth patterns that can be simulated it is 
important, at this time, to summarize Captain Drake's work to include tlie assumptions 
necessary to run the simulation and some of the limitations. 

Many reliabilitv estimators are too mathematically coniplex to allow closed form 
solutions of their properties. In order to compare these properties coniputer simulation 
is one of the onlv techniques available. The simulation constructed bv Captain Drake 
builds a reliability growth pattern and generates test data based on known system reli- 
abilities using the geometric distribution. The discrete reliability growth model being 
evaluated has access to tlus test data for reliabilitv estimation. The test data generated 
by the computer simulation is tvpicallv the number of trials up to and including failure 


and the cause of svstem failure for each predesignated failure in a phase. 
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The inputs required to run the original simulation are [Ref. l: pp. 40-41]: 
1. Number of testing phases. 
2. Number of system failures allowed in each phase. 


Number of possible failure causes. 


Us 


4. Probability of non-occurrence of a particular failure cause in the first phase. 


Reliability growth fraction. 


Un 


6. Discounting option and parameters. 


Several assumptions Were necessary to produce an adequate simulation based on 
these inputs. These assumptions are grouped into two categories -- reliability growth 
pattern assumptions and failure cause assumptions. The assumptions required for simu- 
lating the reliability growth pattern are [Ref. 1: pp. 37-38]: 

1. The reliability growth pattern is non-decreasing. 
2. System reliability changes only at phase boundaries. 


3. Equipment improvements are implemented immediately after a phase ends and 
before any further testing. 


4. Failure causes are corrected only at the end of a phase. 


w 


Each design improvement or repair removes a fixed fraction of the probability of 
reoccurrence for the corresponding failure cause. 


These assumptions narrowed the scope of the reliability growth patterns that could 
be evaluated. The first assumption does not allow the model to track any reliability 
growth patterns that exhibit periods of declining reliability. While for the most part ac- 
tual system reliabilities will be non-decreasing, this restriction does preclude a complete 
evaluation of the reliability estimators under consideration. Assumptions two and three 
are consistent with the previously stated definition of a testing phase. Assumption three 
does not allow modeling of long term design changes that while indicated, may be post- 
poned for whatever reason. Assumption five represents the method that is used in the 
simulation to model the impact of design changes or improvements. According to this 
assumption, any Improvement or design change will remove a fixed fraction (user input 
number five) of the probability of occurrence for a failure cause. The underlving as- 
sumption is that all improvements are equally effective. 

These assumptions were necessary to generate the type of reliability growth patterns 
desired in Captain Drake’s thesis. By introducing the concept of fixed phase reliability 


and modifying the original simulation accordingly, four of these five assumptions were 
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no longer necessary. The exact methodology used to effect this modification will be de- 
tailed 1n the next chapter. 


The last category of assumptions in the original simulation are the failure cause 
determination assumptions. These assumptions are [Ref. 1: p. 39]: 


There exists a finite number of possible failure causes. 
Each failure cause has a fixed probability of occurrence in each phase. 


System reliability can be modeled as a series system of the failure causes. 


£L w N 一 


Each failure cause is stochastically independent of the other failure causes. 


Assumption three dictates the method by which the actual system reliability is 
computed in the simulation. Since the occurrence of a failure cause, by definition, means 
system failure, modeling the system as a series is acceptable. This implies then that the 


actual system reliability may be mathematicallv expressed as: 


R = [JA — P) (2.8) 
S | 


In the above equation P, is the probability of occurrence of the ith failure cause and 
n is the total number of failure causes. 

The simulation is able to generate descriptive statistics about each of the reliability 
growth models being examined. These statistics include the mean, its confidence interval 
and the standard deviation for each reliability estimate at each phase. In order to 
produce these statistics the simulation should be replicated a sufficient number of times. 
The mean and the standard deviation are computed in the usual manner. The confidence 
interval calculation assumes a normal distribution. For the purposes of this paper, 500 
replications of each set of input data were conducted so the normality assumption 
should be reasonable. 

One of the limitations of the model in addition to the restricted number of reliability 
growth patterns that could be analyzed was in the application of the discounting meth- 
ods. The full range of input parameters could not be explored due, in large measure, to 
computer limitations. This limitation typically manifested itself at relatively high system 
reliability. If a failure cause with a very low probability of occurrence caused system 
failure early and if the input parameters for the discounting method were set such that 


a relatively large fraction of this failure was discounted over a relatively short number 


of successful trials then it was possible for this early failure to be discounted to essen- 
tially zero. When the adjusted trials were calculated utilizing Equation 2.3 
(ADJUSTED TRIALS = ACAL TRIALS 


ADIUS ED FAILURE ), 
itv creating a svstem error in the computer. 


then the value approached infin- 


This problem was solved by specifving a lower bound for the adjusted failure. If the 
failure was discounted so that its adjusted value was less than .0000001 then the adjusted 
failure was set to .0000001. This modification allowed the full range of input parameters 
for the discounting methods to be evaluated. The effect on the estimate of system reli- 


ability 1s negligible. 


Il. METHODOLOGY 


One of the major objectives of this paper is to expand the simulation described in 
the previous chapter so that a greater variety of reliability growth patterns could be used 
to evaluate the two reliability growth models. This was accomplished through a modifi- 
cation of the simulation that allowed the user to input the desired system reliabilities at 
each phase. This chapter will detail the method that was used to effect this modification 
along with a description of the reliability growth patterns and failure discounting options 
that were used. 


A. FIXED PHASE RELIABILITY 

One of the limitations of the original simulation was that only constant reliability 
or reliabilitv that increased at a constant rate could be modeled. If a predesignated fail- 
ure cause resulted in a failure then that failure cause's probability of occurrence on any 
succeeding phase was reduced by a predetermined constant amount. Thus, once the user 
of the simulation established the initial inputs for phase one, he had very limited control 
over the actual system reliabilities in the subseguent phases. 

In order to evaluate the two reliability growth models under a greater variety of 
potential reliability growth patterns the concept of fixed phase reliability was introduced 
as an option to the original simulation. With this modification of the program, the user 
has the ability to fix the actual system reliability at each phase. Thus, the reliability 
growth models can be used to estimate system reliability under any reliability growth 
pattern the user desires. 

Incorporating fixed phase reliability into the simulation necessitated an expansion 
of the user-provided input. The basic premise used in computing actual system reliability 
remains the same. This premise is that system reliability is the product of the probabili- 
ties of success of the failure causes ( Equation 2.8, R = Ila — P)). Originally, the 
simulation required that the user input the probability of success for each failure cause 
for the first phase. After that, the fractional improvement factor dictates the actual svs- 
tem reliability in succeeding phases. If the option of fixing the phase reliabilities is cho- 
sen, then the user must specify the probability of success for each failure cause in each 


phase. Equation 2.8 is then applied at each phase to yield the system reliability. 


ii 


The table below illustrates the inputs that would be required to produce a growth 
pattern that initially declines and then increases rapidly. The scenario depicted is a sim- 
ple one involving five test phases with one failure per phase and two failure causes, X 


and Y. Desired reliability is user determined. 


Table 5. FIXED PHASE RELIABILITY EXAMPLE 
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The actual reliability for each phase was computed using Equation 2.8. Consider 
phase one, for example. The desired system reliability 1s 0.80. To achieve this reliability, 
the probabilities of success for failure causes X and Y were input as 0.90 and 0.89 re- 


spectively. This vielded a computed system reliability of: 


n 
R= EC — P) = .90 x .89 = 0.801 
ja=] 

Figure l is a graphical representation of the reliabilities computed in the table. The 
original simulation could not have produced this reliability pattern since declining reli- 
ability is present. This scenario may occur for systems during a portion of their devel- 
opment. While it is true that, in the long run, most systems exhibit non-decreasing 
reliabilities it is important to consider decreasing reliability when evaluating growth 
models. 

In order to invoke this ability to specify reliability at each phase, the number of user 
provided inputs increased from two to ten. If there had been two failures allowed in each 
of the five testing phases then the user inputs would have increased from ten to twenty. 
Therefore, while the option of allowing the user of the simulation to control the actual 


system reliability at each phase significantly increases the variety of different reliability 
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COMPUTED RELIABILITY GROWTH PATTERN 
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Figure 1. Fixed Phase Reliability Example 


growth patterns that can be analyzed, a rather large increase in the volume of input data 


is required. 


B. RELIABILITY GROWTH PATTERNS 

To more fully evaluate the properties and trends of the two reliability growth 
models described in the previous chapter, it was necessary to analyze their performance 
with respect to a variety of different reliability patterns. The eight growth patterns that 
were analyzed in this paper will be presented in this section. 

In conjunction with the reliability growth patterns, another important impact on 
the performance of the reliability estimators was the choice of discounting method. The 


table below depicts the various combinations of parameters for the standard discount 
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method that were analyzed. The discount fraction is the amount of a previous failure 
that is removed each time the discount method is applied while the discount interval is 
the number of successful trials between applications. The Lloyd discount method only 
has one parameter, the confidence interval. For the purposes of this paper, the Lloyd 


method was applied with confidence values of .8 and .9. 


Table 6. STANDARD DISCOUNT METHOD COMBINATIONS 


ie E A 
a ee 
uz a O 
nm S S 
CT EE 
A ER 
5 NA, 
ras T Rer ca 
一 
ras a 















The scenarios used to analyze the reliability growth models in this paper involved 
ten testing phases, i.e., testing was conducted until ten changes had been made. A total 
of five failure causes were considered in each test and a limit of one failure per phase 
was established. The simulation is able to handle any number of test phases, any num- 
ber of failure causes, and any number of failures per phase. The only limitation is the 
capacity of the computer on which the simulation is run. Another practical limitation 
involving the fixed reliability option is the volume of input data that would accompany 
a test design involving numerous phases or failure causes. As can be seen by the reli- 
ability growth patterns and user input tables following, the probabilities of success for 
the failure causes were allowed to fluctuate. The only requirement was that the reliability 
growth pattern established accurately reflected the desired growth pattern. 

Figure 2 and Table 7 describe the first reliability growth pattern. This is not a very 


conventional pattern in that most estimators assume concave growth patterns of the 
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form shown in Figures 5 and 6. However, this type of reliability growth pattern would 
not be unusual in situations where the exact method or technology required to correct 
a lailure-causing defect is not immediately available but, as the system evolves and per- 
sonnel become inore fanuliar with it, the the failure correction process procecds more 
efficiently. 

Figure 3 depicts a decreasing reliability growth pattern. In this instance, an at- 
tempted improvement actually caused system reliability to decrease. This situation, 
which is not effectively addressed by any conventional reliability estimators. could pos- 
sibly occur in experimental or technologically advanced systems where the coinplete 
ramifications of design changes or corrections are not known. 

Figure 4 represents a scenario in which the system under consideration attains a 
moderately high reliability and then stagnates at that level for several phases before 
starting to improve again. This scenario could occur if the exact cause of system failure 
is difficult to assess and would not be unconunon if the system being developed is higlily 
complex. 

Figures 5 and 6 depict conventional reliability growth patterns one would expect to 
encounter when evaluating the majority of systems. In Figure 5, the svstem reliability 
increases rapidly to 0.99 and then remains constant whereas, in Figure 6, the system re- 
liability increases rapidly to 0.90 before beconung constant. It is important in the eval- 
uation of the reliability growth models to compare their performance in relatively 
atvpical, although not uncommon, scenarios such as those shown in Figures 2 through 
5 with their performance in more conventional situations. 

Figures 7, 8, and 9 describe a constant system reliability of 0.90, 0.60, and 0.40 re- 
spectively. While it would be rare to find these system reliabilities occurring in actual 
practice, their inclusion will illuminate some of the trends of the growth models under 
the various failure discounting methodologies. 

Each reliability growth pattern was simulated 500 tunes for each of the failure dis- 
counting combinations described previously. This will enable a more accurate appraisal 
of the capabilities and limitations of the two reliability growth niodels being considered. 
In addition, the most commonly used estimator of reliability, the single phase maximum 
likelihood estimate, will be included on all figures so that a basis for comparing the two 
reliability growth estimators exists. The single phase maximum likelihood estimate, in 


contrast to the maximunı likelihood estimate with failure discounting being evaluated in 
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Figure 3. Pattern II 
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this paper, only considers the number of trials and failures in the current phase. All 
previous data 1s discarded since, presumably, the system reliability fluctuates from phase 
to phase thereby negating the underlying assumption of constant reliability. 

Of the eight reliability growth patterns being simulated, only four (including the 
three cases of constant system reliabilitv, Figures 7, 8, and 9) could have been produced 
with the simulation as originally written. Therefore, the addition of fixed phase reliability 


will enable a more rigorous examination of the proposed reliability estunators. 
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IV. EVALUATION AND INTERPRETATION 


This chapter describes results of the simulation runs for each pattern. The per- 
formance of the models are reduced to graphical form to facilitate comparison. Onlv a 
few of the more representative runs are included in this chapter due to time and space 
constraints. Tle graphical results of many of the remaining simulation runs are in Ap- 
mendix C. 

The analysis is organized according to the reliability growth patterns presented in 
the previous chapter. The performance of cach of the two reliabilitv growth models will 
be evaluated with respect to the growth pattern. The analysis focuses on how well the 
growth model tracked the actual reliability growth pattern. Included in this evaluation 
is an examination of the standard deviatious of the two estimators. 

It 1s not the purpose of this research to conduct enough simulation runs so that 
precise rules can be established for the selection of discounting parameters. This would 
require analvzing many more different tvpes of reliability growth patterns. In addition, 
the choice of discounting methods and their respective parameters is more a function of 
the particular system being evaluated than a function of the reliability growth model. 
Each system will have varymg characteristics as will tlie personnel responsible for con- 
ducting the testing procedures. 

The reliability growth patterns will be referred to bv their respective Roman nu- 
meral designations. These numerals are listed on the figures describing each pattern in 


the previous chapter and are summarized below for completeness: 


1. Pattern I - Convexly increasing reliability. 


2. Pattern II - Reliability initially increasing then decreases for several phases before 
resuming its upward trend. 
3. Pattern III - Reliability increases rapidly to approximately 0.80. Reliability then 


remains constant for several phases before increasing to its final leve] of approxi- 
mately 0.90. 


4. Pattern IV - Reliability increases rapidly to approximately 0.99. 
5. Pattern V - Reliability increases rapidly to approximately 0.90. 
6, Pattern VI - Reliability starts and remains constant at 0.90, 

7. Pattern VII - Reliability starts and remains constant at 0.60. 


8. Pattern VIII - Reliability starts and remains constant at 0.40. 


31 


A. CONSTANT RELIABILITIES -- PATTERNS VI, VII, AND VIII 
1. Standard Discount Method 


RELIABILITY 


Figure 10. 


Figure 10 shows the performance of the two reliability growth models with re- 
spect to Pattern VI. In this particular figure the discount fraction, F, is set at zero (pre- 
vious system failures are not discounted). As one might expect. both models performed 
well. Since the actual system reliability is constant across all phases, the basic underlying 
assumption of constant reliability for the maximum likelihood estimate is valid. There- 
fore, previous data is compatible with current data without the use of failure discounting 
techniques. Thus, including all previous failure data such as done in the Maximum 
Likelihood Estimate with Failure Discounting (MLEFD) will lead to a more accurate 


esumation of reliability. It is also interesting to note that the single phase maximum 
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likelihood estimate discussed briefly in the previous chapter resulted in an estimate of 
0.743 for phase ten. The single phase maximum likelihood estimate considers only the 
number of trials and failures in the current phase and ignores past test data. The ex- 


pected value of this estimate, using Equation 2.5 is 0.744, thus supporting the validity 
of the simulation process. 
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Figure 11. Pattern VÍ, F = 25andl = 6 


Both models did very well in terms of the stability of the estimate when the 
actual system reliability was high. This is largely due to the fact that high system reli- 
ability implies that more test trials will be reguired to produce a failure. More test trials 
translates into additional data for the two models and, therefore, a less variable estimate. 
It should be noted, however, that when actual system reliability is constant, the MLEFD 


does better than the exponential regression model. The standard deviation of the 
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MLEFD is less than half that of the exponential regression estimate. Appendix C also 
contains the graphs of the standard deviations of the two estimates for all eight patterns 


for the interested reader. 
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Figure 12. Pattern VIII, Lloyd method - Cl = .9 


The standard discount method will lead to an overestimation of reliabilitv when 
applied to the MLEFD for the constant system reliability patterns. Figure 11 depicts 
the results of applying this method with the parameter values set at F = 0.25 and I = 
6. While only shghtly affecting the exponential regression model, discounting previous 
failures in this manner resulted in the MLEFD overestimating svstem reliability fairly 
significantly after phase four. This is not unexpected since, as stated before, the as- 
sumption of constant system reliability is valid in this instance. It is important to note, 


however, the sensitivity of the MLEFD model to discount procedures compared to the 
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exponential regression model. Despite using a relativelv small value for F the predicted 
rehabilities were markedly altered. This characteristic of the MLEFD is also evident in 
the patterns to follow. 
2. Lloyd Discount Method 

The Lloyd discount failure discounting method has only one parameter. the 
confidence interval. For the purposes of this study the only values for CI that were ad- 
dressed were 0.80 and 0.90. The formula for applying the Lloyd discount method. 
Equation 2.2, is constructed so that the higher the value of CI, the less a previous failure 
is discounted. Figure 12 graphically represents Pattern VIII and the results of applying 
the Llovd method with CI = 0.90 to the two models. Once again, the effect on the 
MLEFD is rather dramatic with a phase ten estimate of reliability of approximately 0.70 
while the actual system reliability is 0.40. The effect on the exponential regression model. 
although significant after phase seven, is not nearly so severe. This marked etfect of 
applving the Lloyd discount method to the two reliability growth models continued with 
the other patterns as will be seen. Again, the important thing to note is the extreme 
sensitivitv of the MLEFD to failure discounting in general and to the chosen values of 


the discount parameters in particular. 


B. RAPID RELIABILITY GROWTH -- PATTERNS IV AND V 
The discussion of the results achieved for these patterns will center around Pattern 
V. Although the tendencies of the two models were the same for both patterns. the 
graphical portraval of identified trends is better observed with this pattern than with 
Pattern IV where the final systein reliability approaches 0.99. 
1. Standard Discount Method 
Figure 13 depicts Pattern V with the discount fraction, F, set at zero. Both the 
MLEFD and the exponential regression model perform well. It is significant to note, 
however, that the exponential regression model converged to the actual system rehability 
much faster than did the MLEFD. This 1s due in large part to the use of linear regression 
techniques in this model. The standard deviation of the exponential regression was 
comparable to the MLEID for this pattern although still slightly larger. Both the 
MLEFD and the exponential regression model outperformed the single phase maximum 
likelihood estimate thus lending validity to the use of reliability growth models to esti- 
mate system reliability. 
Figure 14 shows Pattern V with F = 0.50 and I = 3 trials. This high a rate of 


failure discounting caused the MLEFD to yield a phase ten estimate of 0.997 when the 
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Figure 13. Pattern V, No Discounting 


actual reliabilitv was 0.900. A correlate of this high estimate of reliability is that the 
standard deviation associated with the MLEFD became verv small relatively quickly. 
The sensitivity of the MLEFD to failure discounting is again apparent in tlus figure. 

The exponential regression model, on the other hand, does remarkably well. It 
quickly approached actual svstem reliability. However, the standard deviation associated 
with this estimate remained high at a relatively constant value of 0.20. So, while the 
mean estimate of system reliability was very good, the associated variance of the esti- 
mate is greater than one would desire. 

The MLEFD performed quite well when F was set at 0.25 and I was put at 15 


as shown in Figure 15. The exponential regression model did slightly overestimate the 
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Figure 14. Pattern V,F = .5 and I = 3 


svstem reliability after the sixth phase however, it was more proficient at matching the 
actual growth of reliability in the system. There was not a significant difference in the 
standard deviations of the two estimates. 

2. Lloyd Discount Method. 

Figure 16 depicts Pattern V with a confidence interval value of 0.80. Note that. 
as on the constant system reliability patterns discussed previously, application of the 
Lloyd discount method in its derived form caused both estimators to predict system re- 
liabilities that were markedly greater than the actual svstem rehabilitv. [his trend was 


consistent no matter what values of CI were used. 
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Figure 15. Pattern V, F = „23 and I = 15 


The conventional estimate of reliability, the single phase maximum likelihood 
estimate, 1s not affected bv discount procedures since only current test data 1s consid- 
ered. The discounting methods addressed in this paper are onlv applied to failures oc- 


curring in previous phases, 


C. DECREASING RELIABILITY - PATTERN Il 
l. Standard Discount Method 
Figure 17 shows Pattern IT with no failure discounting applied. Of special in- 
terest in this graph is the performance of the two reliabilitv growth models during the 
period of declining reliability. The exponential regression model was the onlv model that 


actually demonstrated a decrease in reliability although it still overestimated the actual 


38 





PATTERN V — LLOYD, CI = .8 





RELJABILITY 


=="—= AC|UAL REDAGUMY 
--(]-- MLE SINGLE PHASE 


——V—— EXPONENTIAL REGRESSION 
——+—— MLEFD 


Figure 16. Pattern V, Lloyd method, CI = .8 


reliability during this period. The MLEFD, although slowing in terms of growth rate. 
falled to capture the trend of the actual reliability. The lack of ability to successfully 
predict the downward trend in actual reliability on the part of the MLEFD was present 
regardless of what values were assigned the discount parameters. In fact, as one might 
expect, discounting previous failures exacerbated the problem. 

Figure 18 shows Pattern II with F = 0.75 and I = 6. The exponential re- 
gression model did very well, particularly during the phases where system reliability was 
declining. The MLEFD, in contrast, performed poorly overestimating system reliability 


significantly after phase three and failing to indicate that system reliability decreased. 
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Figure 17. Pattern If, No Discounting 


The standard deviation of the exponential regression estimate still exceeded that 
of the MLEFD. This characteristic is fairly consistent for all the patterns that were ex- 
amuned in the course of evaluating these two models. It should be noted, however, that 
a small standard deviation does little good if the estimator is yielding markedly incorrect 
predictions. 

2. Lloyd Discount Method 

Figure 19 shows Pattern II with CI = 0.90. The application of the Lloyd 
method profoundly affected the performance of the MLEFD while only slightly altering 
the exponential regression model. This method of failure discounting did cause both 


models to overestimate svstem reliability by phase nine but, again, had less of an effect 
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Figure 18. Pattern IL, F = 75,1 — 6 


on the exponential regression model. It is especially noteworthy to observe the cflect of 
the Llovd method on the MLEFD during the period of decreasing svstem reliability. The 
real sensitivity of the MLEFD to discount methods and parameters is particularly evi- 


dent in this pattern. 


D. INTERMITTANT RELIABILITY GROWTH - PATTERN Hi 
1. Standard Discount Method 
Figure 20 depicts the performance of the two models when no failure dis- 
counting is applied. The exponential regression model outperformed the MLETD at ev- 


ery phase. Again, the exponential regression model succeeded in capturing the trend of 
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Figure 19. Pattern II, Lloyd method, CI = .9 


the reliability growth pattern from phase four to phase siv whereas the MLEFD cxhib- 
ited a constant rate of increase. 

When the discount parameters F and I were set at 0.25 and six respectively, the 
NICEFD did much Detter as can be seen m Figure 21. Both miedels conterzed tONNE 
correct prediction of system reliability by plase ten. Llowever, the exponential regression 
model converged to the correct value mucl more quickly than did the The 
standard deviations of the two estimates were not too significantly different [rom each 
other for this particular pattern and discount method. 

The performance of the two models on reliability growth patterns II and III 
demonstrate one of the inherent advantages of the exponential regression methodologv. 


Since this method makes use of the techniques of linear regression, it is better able to 
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Figure 20. Pattern III, No Discounting 


track changing system reliability than is the MLEFD. This attribute of the exponential 
regression model will be developed further in the next chapter. 
2. Lloyd Discount Method 

Figure 22 shows how the models performed using Lloyd failure discounting 
with CI = 0.90. Again, applying the Lloyd method caused the MLEFD to overestimate 
system reliability after phase four. Some slight overestunauon did occur in the expo- 
nential regression model during phases nine and ten however, the overall effect was to 
enable the model to better track actual svstem reliability during the middle phases. 
During the discussion of the discounting methodologies in Chapter II, it was noted that 
the Lloyd failure discount method is applied after every trial that did not result in the 


reoccurrence of a previous failure cause. This means of application is a likely source of 
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figure 21. Pattern lII, F = 25,1 = 6 


the overestimation characteristics of both models when Lloyd discounting is applied. 


An idea on a possible remedy to this problem will be presented at the end of this chapter. 


E. CONVEX RELIABILITY GROWTH - PATTERN I 

Most reliability growth patterns are thought to be essentially concave in shape. 
much like Patterns IV and V. This is normally a valid assumption in that major im- 
provements in the reliability of a new system most often occur during the early stages 
of development as the obvious flaws in the design and manufacture become apparent. 
[tis not inconceivable, however, to experience a period of convex growth such as shown 
in Pattern I. One would want to have confidence in the chosen reliability estimator to 


correctly estimate system reliability regardless of its actual form. 
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Figure 22. Pattern III, Lloyd method, CI = .9 


1. Standard Discount Method 

Figure 23 shows the performance of the two models with no failure discounting 
being applied. It is apparent that, even without the benefit of failure discounting tech- 
niques, the exponential regression model does well. It is able to reflect the trend of actual 
system reliabilitv growth and the actual estimates are fairly close to the true system re- 
liability. The MLEFD does not do so well and, in fact, is even outperformed bv the 
single phase maximum likelihood estimate in the later phases. One would expect in this 
case to have significant results when failure discounting ts applied due to the demon- 


strated sensitivity of the MLEFD to this procedure. 
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Figure 23. Pattern I, No Discounting 


Figure 24 depicts Pattern I with F = .75 and I = 6. Although failing to capture 
the shape of the actual reliability pattern, the phase estimates produced by the MLEFD 
are quite good. Fractionally removing 75 percent ofa previous failure after six successful 
trials is a substantial discount and one would expect that such a large discounting 
scheme would lend itself more to the conventional concave reliabilitv growth patterns. 
This particular type of scenario would benefit greatly by the ability to alter the discount 
parameters during the course of the test. Initially, one would not want to discount pre- 
vious failures by a large amount since actual svstem reliability is only improving mar- 
ginally. As the system evolves and reliability begins to make substantial jumps then 


previous failures should be discounted more. Of course, this presumes a knowledge of 
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Figure 24, Pattern I, F = .75 and I = 6 


the actual svstem reliability in order to properly design sucli a plan and this is never the 
case. However, in depth knowledge of the proposed systein could indicate the expected 
shape of the reliabilitv pattern and this could be made use of in test design. Although 
changing the discount parameters during the course of the test is possible in actual 
practice it is not, unfortunately, possible to do with the simulation in its current form. 
The parameters for the discount procedures remain constant once they are input at the 
beginning of the simulation. 
2. Lloyd Discount Method 
Figure 25 shows the Lloyd method with CI = 0.90. The exponential regression 


model maintains the shape of the true system reliability while converging to the actual 
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Figure 25. Pattern I, Lloyd method, CI = .9 


reliability by phase eight. The MLEFD becomes concave in shape much as it did for the 
standard discount method shown in Figure 24. However, the intermediate phase reli- 
ability estimates are very good. The standard deviation of the MLEFD continues to be 
less than that of the exponential regression model although, as shown in Chapter II, the 
mean square error for the MLEFD exceeds that of tlie exponential regression model. 
The Lloyd method does better with this pattern than with any other. Previ- 
ously, application of this method caused both models to overestimate system reliability 
quite significantiv. In this case, using this method improved the performance of the ex- 
ponential regression model and, although distorting the shape, resulted in improved re- 


hability estimates from the MLEFD as well. 
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F. LLOYD FAILURE DISCOUNTING ( REVISITED ) 

Applving the Lloyd failure discounting method in its current form consistentlv 
caused both models to overestimate the true system reliabilitv. This was true for the 
fixed phase reliabilitv growth patterns that were addressed in this paper as well as for the 
growth patterns that were analyzed in Captain Drake’s work previous to this (Ref. ]: pn. 
64-65]. One potential source of this problem is that the Lloyd method is applied after 
every successful trial. 

The formula for computing adjusted failures using the Lloyd method is given by 
Eaton 2.2 repeated below: 


1 
PODOSTEDFAILURE=1—(1-CDT fT>0. (202) 


If T equals zero then the adjusted failure value is set to one. This formula is used to 
compute adjusted failures after each trial in which the cause for a previous failure did 
not reoccur. Since the problem appears to lie in the number of times the method is ap- 
plied the inclusion of an interval may lead to more accurate results. 

A modified version of the Lloyd discount method which employs an interval of 


application is as follows: 


1 
ADJIUSTED FAILURE= 1- (1- CDM ifM 0 (4.1) 


where, in this case. 


k T 

M = IN KT DT? 
LDI is defined as the Lloyd Discount Interval and becomes another parameter that can 
be altered by the user of the simulation. If M equals zero then the adjusted failure value 
is set at one as before. By its construction, one can observe that for all values of T less 
than the predesignated LDI, then the value of M will be zero. M will increase bv one for 
each group of successful trials meeting the LDI. Note also that specifving an LDI of one 
allows the user of the simulation to employ the Llovd method in its original form. 

Some preliminary runs of the simulation were made with this modification included. 
The results were encouraging. Figure 12 examined previously illustrated the eflect of the 
Lloyd method on the two models when system reliability was actually a constant 0.40. 
Figure 26 shows the result of applying this modified version of the Lloyd method with 
the value for CI remaining at 0.90 and the LDI set at eight. Both the MLEFD and the 
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Figure 26. Pattern VIII, Modified Lloyd Method, CI = .9, LDI = 8 


exponential regression model performed much better than before. The LDI value of 
eight resulted in similar improvements for the other constant svstem reliability patterns. 

The results on the other patterns were similar to those represented in Figure [7 but 
the value for the LDI had to be adjusted for each pattern. Before any specific conclu- 
sions can be drawn about the benefits of this modification many more computer runs 
and more in depth analysis is required. However, based on these preliminary results, the 


inclusion of an interval does appear to have some merit. 


V. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS 
A. SUMMARY 


lhe purpose of this paper was to evaluate two discrete reliability growth models. 
Each of these models has the ability to use fractionally discounted failure data to make 
an estimate of the actual system reliabilitv at any point during the conduct of a test. 
Fractionally discounting failures occurring early in a test procedure allows the data ac- 
cumulated during the initial stages of a test to be used in current reliability estimates. 
Therefore, if the discounting procedure is performed wisely, more accurate reliability 
estimates with a higher degree of confidence should result. 

Two procedures for discounting previous systern failures were described in Chapter 
II. The standard discount method requires the user to specify two parameters, the frac- 
tion of failure to be removed and the discount interval. The fraction of failure to be re- 
moved, referred to as the discount fraction or F, specifies how much a previous failure 
is reduced upon each application of the discount procedure. The discount interval, I, is 
simply the number of successful trials (trials without a reoccurrence of the particular 
failure cause) that must occur between applications of the procedure. The other failure 
discounting method was proposed by David Lloyd and is referred to as the Lloyd dis- 
count method. This method utilizes only one input parameter, the level of confidence 
for an upper confidence interval on reliability. This method, in its original forim, is ap- 
plied after every successful trial. A modified Lloyd discounting method which employs 
an interval of application was derived in Chapter IV. 

Two discrete reliability growth models were described. The first model is a form of 
the maximum likelihood estimate with the addition of failure discounting. Since the 
maximum likelihood estimate assumes a constant reliabilitv between phases, the addition 
of failure discounting will enable this model to estimate changing system reliability. The 
Maximum Likelihood Estimate with Failure Discounting, or MLEFD, estimates system 
reliability as the ratio of the number of successful trials to the total number of trials with 
this ratio being computed after application of failure discounting. 

The other model was developed by H. Chernoff and W.M. Woods and 1s a deriva- 
tion of an exponential single phase reliability estimate. This model employs linear re- 
gression techniques to estimate reliabilitv after any change has been made to the unit 


of hardware under test. Because of this, the exponential regression model is able to track 
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changing svstem reliabilitv without the benefit of failure discounting although the use 
of failure discounting can enhance the model's performance. 

The means of evaluating these two models was a Monte Carlo simulation. The 
simulation used was originally written by Captain James Drake and was modified so that 
fixed phase reliabilities could be modeled. The fixed phase reliability modification allows 
the user of the simulation to evaluate the performance of the two reliability growth 
models on any pattern of actual reliability growth desired. Thus, periods of declining 
reliability, periods of no reliability growth, and convexly growing reliability among oth- 
ers can be easily modeled and the performance of the two growth models analvzed. 

Eight varying reliability growth patterns were analyzed. Both of the reliability 
growth models were evaluated against these eight patterns and under 13 different failure 
discounting combinations including no failure discounting. Each model, pattern and 
failure discounting combination was replicated 500 times. The abilitv of each model to 
accuratelv track the actual svstem reliability was evaluated along with an examination 


of the stability of the estimate produced by the model. 


B. CONCLUSIONS 

The evaluation of the two reliability growth models on the eight different reliability 
growth patterns leads to several general conclusions. The most obvious conclusion 1s 
that the use of either one of these growth models is superior to the standard single phase 
maximum likelihood estimate. Both growtl models were far more accurate in terms of 
estimating the true system reliability and the variance associated with the model esti- 
mates was less than that produced by the single phase MLE. 

The MLEFD is particularly sensitive to the choice of failure discounting procedure 
and parameters. This sensitivity is made use of in order to allow the model to track 
changing reliability. However, this property also places a premium on the ability of the 
test engineer to correctly select the proper discounting method and parameters. 

The MLEFD did verv well when the actual system reliability was constant. Con- 
stant system reliability is seldom present in actual reliability testing, however. The 
MLEFD also did well when the true reliability was increasing at a constaut rate. Al- 
though the exponential regression model more closelv followed the actual reliability 
growth pattern, the MLEFD would be preferred in this instance because of its smaller 
variability. The MLEFD did best with these patterns when the standard discount 
method was used with the discount fraction equal to 0.25 and the discount interval equal 
to 6 or 15, 


The exponential regression model, although more variable in the earlv stages of 
testing, demonstrated the ability to accuratelv track each of the eight reliability growth 
patterns evaluated, even without the use of failure discounting. This mode] is also much 
less sensitive to the choice of discount method although its performance can be some- 
what enhanced through failure discounting. The MLEIFD, on the other hand, failed to 
track the decline in reliability in Pattern II. > 

Therefore, the exponential regression model is recommended for situations where 
a proven means of failure discounting does not exist. If one is confident of the failure 
discounting methodology then the MLEFD may be more advantageous due to the sta- 
bility of the estimates produced by it. Regardless of the degree of confidence one has in 
the failure discounting procedures, the exponential regression model is recommended if 
abnormal rehability growth is anticipated. Abnormal reliability growth can be defined 
as a reliability pattern that has periods of decreasing or constant reliabilitv in the middle 
of a generally increasing growth rate. The exponential regression model demonstrated 
superiority to the MILEFD in its ability to track differing patterns of growth. 

The Lloyd discount method, in its original form, caused both models to overesti- 
mate the actual system reliability. Therefore, use of this method with these two reliability 
growth models is not recommended. A modified version of the Llovd method which 
emplovs a discount interval demonstrated superior performance in some patterns to the 


original Lloyd discounting method. 


C. RECOMMENDATIONS FOR FURIHER STUDY 
The following are recommendations for further studv and possible improvement to 


the models introduced in the study: 


s This study addressed only discrete reliabilitv growth models. A similar study ana- 
Ivzing the characteristics of continuous reliability growth models should be con- 
ducted. This would necessitate the construction of a different simulation. 


e Currently, the failure discounting mechanism used in the simulation emploved in 
this analysis only allows the user to set parameter values at the beginning. This 
should be modified so that discount parameters can be changed during the course 
of the simulation. 


e The failure discounting mechanisin should be modified so that different parameters 
can be applied to different failure causes. It may be desired that system failures 
caused by failure cause A, for example, be discounted less than those caused by 
failure cause B. This is not possible with the current simulation. 


e Further analysis is required of the modified Lloyd failure discounting method. 
Many more simulation runs with varyiug reliability growth patterus and dillerent 


combinations of discount parameters are required before any solid conclusions can 
De reached 


A weighted least squares method that weights recent data more heavilv than pre- 
vious data should be analvzed to determine its ability to track reliability growth 
patterns with sudden changes in slope or direction. 
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APPENDIX A. DERIVATION OF THE EXPONENTIAL REGRESSION 
MODEL 


This model was developed by H. Chernoff and W.M. Woods. The derivation has 
been presented in previous works, most recently by W.M. Woods in a paper entitled 
“Reliability Growth Models” and by Captain James Drake in a thesis entitled “Discrete 
Reliability Growth Models using Failure Discounting”. The exponential regression 
model ts based on the exponential single phase reliability estimate. The exponential sin- 


gle phase estimate mav be expressed mathematically as follows: 

2 — q 

The exponential regression model uses linear regression to estimate A so 
"NS 4 + bk 


where k is the testing phase being used to compute system reliability. Thus, the expo- 


nential regression model has the following formula: 
= | CCO ZS VA 


The estimates, x, and By, for x and f at the end of the kth phase are obtained using 
the techniques of linear regression and an unbiased estunate for (a + fk). Let F denote 
the total number of failures possible during the kth phase and let j equal the failure 
number in phase k such that j = 1,2,3....,F, V, eguals the number of trials between the 
(j-1)st failure and the jth failure, including the jth failure, in the kth phase. 


An unbiased estimate of (a + $k) using the jth set of trials in phase k is given by: 


| 


Ii ,=1+ we 


+ + je Z 


L 
2 
IPN, = I (the first test was a failure) then F,, 15 set equal to zero. The least squares 


estimates, x, and £,, for « and f at the kth phase are then: 
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Substituting these estimates for a, and f, into the reliability equation vields the following 


estimate of reliability for phase k: 


R, = 1 e TP fork > 4 
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APPENDIX B. FORTRAN CODE FOR SIMULATION 


Jao aa ooo da as ss so son ats „Jauno outono a! 

TYS RTT RETER Trede dede dele ete do dodo vodovode vodo ve de do de dedo dok do dodo do do de dovo se doslo do dodo dodo dodo do dd ded iv ie 
Ja 

PAT 


ci DISCRETE RELIABILITY GROWTH SIMULATION 7 
* PROGRAMIIED BY JAMES E DRAKE = 
6 MODIFIED BY JAMES D CHANDLER ES 
É LAST MODIFIED 29 FEB 1988 7 
* THE FOLLOWING EXTERNAL FILES ARE USED BY THE PROGRAM S 


* INPUT : DATA AND PARAMETER INPUT FILE (DEVICE # 10) 3 
* THESIS : OUTPUT FILE CONTAINING INTERMEDIATE COMPUTATIONS * 
žē (DEVICE ; 20) 


*  RELIAB: OUTPUT FILE CONTAINING FINAL RESULTS OF THE SIMULATION - 
% DEVICE 7350) * 
EST : OUTPUT FILE CONTAINING EACII PHASE ESTIMATE FOR EACH * 
z REPLICATION OF THE WOODS WEIGHIED AVERAGE ESTIMATE 3 
* (DEVICE # 40) * 
* MLEWD : OUTPUT FILE CONTAINING MLE ESTIMATES USING DISCOUNTING BS 
E FOR EACH PHASE AND EACH REPLICATION ES 
* (DEVICE # 50) + 
OMMI(LESP O: OUTPUT FILE CONTAINING MLE ESTIMATE FOR EACH SINGLE PHASE  * 
SE AND ALL REPLICATIONS USING NO DISCOUNTING ni 


- (DEVICE # 60) 

o. REGEST : OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 
på REPLICATION OF THE EXPONENTIAL REGRESSION ESTIMATE 
N (DEVICE 4 70) 


, s a a af 
we 6 OG Å 


* THE FOLLOWING IS A LIST OF KEY ARRAYS USED IN THE SIMULATION 


* A : MAIN WORKING ARRAY CONTAINS PROBABILITY OF SUCCESS FOR vs 
Je EACH FAILURE CAUSE, NUMBER OF TRIALS UNTIL FAILURE FOR x 
x EACH FAILURE CAUSE AND THE SYSTEM, CAUSE OF FAILURE, se 
e PHASE NUMBER, ADJUSTED NUMBER OF TRIALS AND ADJUSTED ve 
* NUMBER OF FAILURES de 
x DIMENSION ( ((2*/CAUSES)+7),/FAILURES ) * 
* NFAPH : CONTAINS THE NUMBER OF FAILURES IN EACH PHASE * 
še DIMENSION (1,##PHASES) = 
* NFCAUS : BINARY ARRAY USED TO DETERMINE IF A FAILURE OCCURRED IN * 
* A PHASE : 
* DIMENSION ( 1,#FAILURE CAUSES) ze 
* NTRIAL : CONTAINS THE NUMBER OF TRIALS SINCE LAST FAILURE OR zs 
že DISCOUNTING FOR EACH FAILURE CAUSE * 
še DIMENSION ( 1,#FAILURE CAUSES ) * 
%  PHREST : RECORDS THE PHASE ESTIMATE FOR EACH ESTIMATOR WITHIN A å 
x SINGLE REPLICATION * 
ve DIMENSION (4,;PHASES) te 
e ROW 1: WOODS WEIGHTED AVERAGE ESTIMATE ve 
* ROW 2 : MLE WITH DISCOUNTING # 


Sy 


ROW 3 : SINGLE PHASE MLE de 


ROW 4 : EXPONENTIAL REGRESSION ESTIMATE % 
AREL : CONTAINS ACTUAL SYSTEM RELIABILITY IN EACH PHASE x 
DIMENSION (1,;PHASES) % 
YJK : CONTAINS YJK VALUES UP TO 1000 Y 


DIMENSION (TOCO) ve 
CUMSF : CONTAINS THE NUMBER OF SUCCESS AND FAILURES FOR EACH as 
FAILURE CAUSE (USED WITH WOODS WEIGHTED AVERAGE EST. ) v 
DIMENSION (3,/FAILURE CAUSES ) _ 
ROW 1 : NUMBER OF FAILURES ve 
ROW 2 : NUMBER OF SUCCESSES ve 


RÓW 3 : ADJUSTED NUMBER OF SUCCESSES N 
REG : ARRAY USED TO COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE * 
DIMENSION (5,#PHASES) 7 
ROW 1: K BAR ve 
ROW 2 Y BAR ve 
ROW 3 Y BAR FOR THE PHASE ve 
ROW E TABA se 
ROW 5 A HAT ve 
THE REMALNING ARRAYS ARE USED TO COMPUTE THE MEAN AND VARIANCE JE 


OF EACH ESTIMATE AT EACH PHASE. THEY ALL HAVE THE SAME DIMENSIONS E 
AND STRUCTURE w 
DIMENSION (4,;#PHASES) = 
RÓW 1 : RUNNING SUM OF ESTIMATES 


ROW 2 : RUNNING SUM OF SQUARED ESTIMATES Je 
RÓW 3 : MEAN OF THE ESTIMATES * 
RÓW 4 : STANDARD DEVIATION OF THE ESTIMATES x 
EST : VALUES FOR THE WOODS WEIGHTED AVERAGE ESTIMATE 26 
MLEWD : VALUES FOR THE MLE WITH DISCOUNTING * 
MLESP : VALUES FOR THE SINGLE PHASE MLE * 


REGEST : VALUES FOR THE EXPONENTIAL REGRESSION ESTIMATE 


sesede sete a V V V O NE O O O O O VE EEE IEEE Tennant 


DEFINE AND DIMENSION VARIABLES 


PARAMETER (NR=50,NC=200) 

INTEGER REP,CUMSF,DISOPT,FRELOP,LDI,ALD 

REAL*4 MIN 

REAL*8 DSEED, MLESP NZEWDSESNCENE 

DIMENSION NFAPH(NR) ,A(NR,NC) ,NECAUS(NR) „,NTRIAL(NR) ,PHREST(4,NR) ,ES 
CT(4,NR) ,MLEWD(4,NR) ,MLESP(4 ,NR) ,REGEST(4 ,NR) ,AREL(NR) ,YJK( 1000) „GU 
CMSF(3,NR),REG(5,NR) 


READ IN THE NUMBER OF CAUSES TO BE USED ( NCAUSE ) AND THE NUMBER 
OF PHASES ( NPŇÁSE ) EJ E 


REĀD(10,*) NCAUSE 
READ(10,*) NPHASE 


CHECK IF FIXED RELIABILITY OPTION IS DESIRED. FIX EULER'S NUMBER. 


READ(10,*) FRELOP 
EUL = 0.5772156648 


C CREATE VARIABLES FOR THE ROW INDICES OF THE WORKING MATRIX ( A ) 

C IPHASE: PHASE 

C ISYSPR: ACTUAL COMPONENT RELIABILITY 

C INTR: NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 

C IFAILC: CAUSE OF THE FAILURE 

C IADJF: ADJUSTED NUMBER OF FAILURES ED 
C AFTER DISCOUNTING HAS BEEN APPLIED 

C IADJT: ADJUSTED NUMBER OF TRIALS AFTER DISCOUNTING HAS BEEN APPLIED 

C IYJK: YJK COMPUTED ON THE ADJUSTED NUMBER OF TRIALS 


IPHASE = (2*NCAUSE)+1 
ISYSPR = IPHASE +1 
INTR = ISYSPR + 1 
IFAILC = INTR+ 1 
Ange = IFATLO + 1 
IADJT < IADJF + 1 
IYJK < IADJT t 1 


C READ IN THE NUMBER OF FAILURES IN EACH PHASE ( NFAPH(I) ) AND 
C COMPUTE THE TOTAL NUMBER OF FAILURES IN THE TEST ( NFAIL ) 


NFAIL = 
DO 10 I=1,NPHASE 
READ(10,*) NFAPH(I) 
NFAIL = NFAIL + NFAPH(I) 
10 CONTINUE 


C INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRIAL FOR EACH CAUSE 
C IN EACH PHASE IF FRELOP EQUALS ONE. 


IF (FRELOP .EQ. 1) THEN 
DO 15 I=1,NCAUSE 
DO 15 J=1,NFAIL 
READ(10,%) A(1,J) 
15 CONTINUE 
ELSE 


C INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRIAL FOR EACH CAUSE 
C IN THE FIRST PHASE IF FRELOP EQUALS ZERO. 


DO 20 I=1,NCAUSE 
READ( 10,*) A(I,1) 
20 CONTINUE 


ENDIF 


INPUT THE REMAINING VARIABLES , THE NUMBER OF SUCCESSFUL TRIALS 
BEPFOREFZFDISEOUNŃ TS APPLIED (N); THE DISCOUNT FACTOR (R); THE SEED 
FOR THE RANDOM NUMBER GENERATOR, GGUBFS, (DSEED); RELIABILITY 
GROWTH FRACTION (FRIMP); TRIGGER FOR PRINTING INTERMEDIATE OUTPUT 
ČIOPT) 


ci 6161 CIC) 
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6361011611061 CHE 


C 


IOP TIS 
10012. 
Or lem 
IOPT4 : 


READ( 10,*) 
READ( 10,*) 
READ( 10,*) 
READ(10,*) 
READ( 10 ,*) 
READ(10,*) 
READCIO +) 
READ( 10,7%) 
READ( 10,*) 
READ(10,%) 
READ(10,%) 
READ(10,%) 
READ( 10,*) 


DISCOUNTING OPTION TRIGGER (DISOPT); 
PARAMETER (GAMMA); 


N 

R 
DSEED 
FRIE 
NREP 
DE 
TORII 
IOPT2 
TORES 
IOPT4 
DISOPT 
GAMA 
LDI 


XNREP = NREP 
DSEED1 = DSEED 


C OF EACH ESTIMATOR 


30 


DO 30 J=1,NPHASE 
DO 20- 
EST(I,J) = 
MLEWD(I,J) 
MLESP(I,J) 
REGEST(I,J) 
PHREST(I,J) 
CONTINUE 


TRIGGERS FOR SAVING EACH ESTIMATE AT EACH PHASE FOR EACH ESTIMATOR 
WOODS WEIGHTED AVERAGE MODEL 

MLE WITH DISCOUNTING 

SINGLE PHASE MEE 

EXPONENTIAL REGRESSION MODEL 


LLOYD FAILURE DISCOUNTING 


LLOYD DISCOUNT INTERVAL 


INITIALIZE THE ARRAYS USED TO COMPUTE THE MEAN AND STANDARD DEVIATION 


C COMPUTE AND STORE THE YJA VALUES UP 10 [00 


40 


VIE) = ORO 


DO 40 1-1, 


999 


YIKO FEN OT 


CONTINUE 


C COMPUTE AND STORE K BAR FOR THE EXPONENTIAL REGRESSION MODEL 


50 


C 


SUM = 0.0 
DO 50 I=1, 


NPHASE 


SUM = SUM + I 
REG(1,I) = SUM/I 


GONTINUE 


BO 500 REF=INNRUP 


MAJOR REPETITION OF THE SIMULATION LOOP 


60 


C INITIALIZE FAILURE CAUSE VECTOR (NFCAUS) AND (CUMSF) 
UC IPJIE THE INITIAL SYSTEM RELIABILITY 


REL = 1. 
DO 60 I=1,NCAUSE 
NFCAUS(I) = 0 
REL = REL + A(1,1) 
DO 60 J=1,3 
CUMSF(J,I) = 0 
60 CONTINUE 


C INITIALIZE COLUMN (FAILURE # ) COUNTER FOR THE WORKING ARRAY (A) 
J = 1 


C LOOP TO COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 
C AND THE CAUSE OF FAILURE FOR EACH FAILURE IN EACH PHASE 


DO 130 K=1,NPHASE 


C SKIP ACTUAL COMPONENT RELIABILITY COMPUTATION AFTER FIRST REP 
C AND FOR FIRST FAILURE 


KE EU: 1) CUTE 75 
W CREER. GT. 1) GOTO 75 
REL = 1. 


U IE FIXED RELIABILITY OPTION IS SELECTED THEN PHASE RELIABILITIES 
C ARE COMPUTED AS FOLLOWS 


IF (FRELOP .EQ. 1) THEN 
DO 65 I=1,NCAUSE 
REL = REL*A(I,J) 
NFCAUS(I) = 0 
65 CONTINUE 
ELSE 


C COMPUTE NEW ACTUAL RELIABILITY FOR THE COMPONENT IN PHASE K 
DO 70 I=1,NCAUSE 


C INCREASE CAUSE PR(SUCCESS) IF IT CAUSED FAILURE IN THE PREVIOUS PHASE 
C COMPUTE NEXT PHASE RELIABILITY AND REINITIALIZE NFCAUS (NOT USED IF 
C FIXED PHASE RELIABILITY OPTION IS SELECTED). 


IF(NFCAUS(I).EQ. 1) THEN 
MEN NOE AO I )*FRIMP) 
ELSEIF(NFCAUS( I). NE. 1) THEN 
BOLA = AcI;5(J-1)) 
ELSE 
ENDIF 
REL = REL*A(I,J) 
NFCAUS(I) = 0 
70 CONTINUE 
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ENDIF 


(5 Jl = 1 
TRTOT = 0.0 


C COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE AND THE 
C CAUSE OF FAILURE FOR EACH FAILURE IN THE PHASE 


DO 120 L=1,NFAPH(K) 
IF(REP.GT.1) GOTO 90 
IF(J1.EQ. 1) GOTO 85 
IF (FRELOP .EQ. 1) GOTO 85 
DO 80 I=1,NCAUSE 

AĢISS DACI. (J-1)) 


80 CONTINUE 

85 A(ISYSPR,J) = REL 
A(IPHASE,J) = K 

90 MIN = 7.2E75 


DO 110 I=1,NCAUSE 
C ASSIGN # TRIALS FOR CAUSES WITH PR(SUCCESS) = 0 OR 1 


IF(A(I,J).GE. 1.) THEN 
A((I+NGAUSE) ,J) = 7. 2E75 
GOTO 100 

ELSEIF(A(I,J). EQ. 0.) THEN 
A(CI+NCAUSE),J) = 1. 


GOTO 100 
ELSE 
ENDIF 
C CONVERT UNIFORM (0,1) RANDOM VARIABLE TO GEOMETRIC (4 TRIALS UNTIL 
C FAILURE ) FOR EACH FAILURE CAUSE. RECORD THE MIN # TRIALS FOR THE 
C CAUSES AS THE SYSTEM # TRIALS UP TO AND INCLUDING FAILURE AND 
C RECORD THE FAILURE CAUSE 
A((I+NCAUSE),J) = INT(1. +( LOG(GGUBFS( DSEED))/LOG( A(1,J)))) 
100 IPĻĀCĢ ITNGAUSEJ.J) LE. MIN) MIHE 
MIN = A((I+NCAUSE),J) 
IMIN = I 
ELSE 
ENDIF 
110 CONTINUE 


ACIFAILC,J) = IMIN 
NFCAUS( IMIN) = 1 


C GOMPUTE THE TOTAL ¿¿ OF TRIALS FOR THE MLE SINGLE PHASE ESTIMATE AND 
C INCREMENT FAILURE # COUNTERS 


ACINTR,J) = MIN 
TRTOT = TRTOT + ACINTR,J) 
J = J + 1 
Ji = JS 
120 CONTINUE 


C COMPUTE THE MLE ESTIMATE OF COMPONENT RELIABILITY FOR THIS PHASE AND 
C COMPUTE THE RUNNING SUM OF ESTIMATES AND THE SUM OF ESTINATES SQUARED 
C FOR COMPUTATION OF THE MEAN AND STANDARD DEVIATION OF THE ESTIMATE 


PHREST(3,K) = (TRTOT - NFAPH(K))/TRTOT 
HEB er Clk) Meron Gly keer tie ig. Kk) 
MEESEC2,K) NEESENZSKJS PCI IN 12) 


130 CONTINUE . 


C INITIALIZE THE ADJUSTED NUMBER OF FAILURES TO 1 AND THE COUNT OF THE 
C NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING (NTRIALS(I) ) TO 0 
C IN PREPARATION FOR THE DISCOUNTING ROUTINE 


DO 140 J=1,NFAIL 
A(IADJF,J) = 1. 
140 CONTINUE 


DO 150 I=1,NCAUSE 
NTRIAL(I) = 0 
150 CONTINUE 


DISCOUNTING ROUTINE REVIEWS ALL PAST FAILURES AND CAUSES TO DATE 
AND DETERMINES IF THE DISCOUNTING CONDITIONS HAVE BEEN MET. COMPUTES 
THE ADJUSTED FAILURES, THE ADJUSTED # OF TRIALS AND YJK 


Ca tato) 


J=0 
DO 300 K=1,NPHASE 
DO 200 L=1,NFAPH(K) 
EJI 


C UPDATES THE NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING FOR EACH 
C FAILURE CAUSE 


ICAUSE = INT(A(IFAILC,J)+. 5) 
DO 160 I=1,NCAUSE 
IF( ICAUSE. EO. I) THEN 
NTRIAL(I) = 0 
ELSEIF( ICAUSE. NE. I) THEN 
NTRIAL(I) = NTRIAL(I) + INT(A(INTR,J)+. 5) 
ELSE 
ENDIF 
160 CONTINUE 


200 CONTINUE 
C CHOOSE DISCOUNTING METHOD TO BE USED 
IF(DISOPT. NE. 2) GOTO 180 
C PERFORM LLOYD" S FAILURE DISCOUNTING METHOD 
KW 171.7 
Il = INT(ACIFAILC,I)+. 5) 


IF(NTRIAL(I1).EQ.0) THEN 
ACIADJF,I) = 1.0 
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GOTO 170 
ELSE 
ENDIF 


C THIS IS THE MODIFIED LLOYD METHOD USING A DISCOUNT INTERVAL. THE 
C ORIGINAL DISCOUNT METHOD MAY BE EMPLOYED BY SETTING LDI TO ONE. 


ALD = INT(NTRIAL(I1)/LDI) 
IF(ALD .EQ. 0) THEN 
A(IADJF,I) = 1.0 
GO TO 170 
ELSE 
A(IADJF,I) = 1.0 - ((1.-GAMA)**(1.0/ALD)) 
ENDIF 


170 CONTINUE 
GOTOTZ IO 


C PERFORMS STRAIGHT PERCENT FAILURE DISCOUNTING AND 
C COMPUTES THE ADJUSTED # OF FAILURES 


180 DO 190 I=1,J 
Ji = INTCAC IEA REG) ES) 
IF(NTRIAL( 11). EQ. 0) THEN 
ACIADJF,I) = 1. 
ELSEIF(NTRIAL(I1).GE.N) THEN 
ACIADJF ,I) = A(IADJF,I)*((1. -R)**(NTRIAL(I1)/N)) 
ELSE 
ENDIF 
190 CONTINUE 


C ADJUSTS THE # TRIALS SINCE FAILURE OR DISCOUNTING FOR THOSE CAUSES 
C THAT HAVE MET OR SURPASSED THE DISCOUNTING THRESHOLD 
C FOR THE STRAIGHT PERCENT DISCOUNTING METHOD 


DO 205 I=1,NCAUSE 
IF(NTRIAL(I).GE.N) NTRIAL(I) = MOD(NTRIAL(I),N) 
205 CONTINUE 
210 TADJT = 0.0 
TYJK = 0.0 
TPYJK = 0.0 
Kl = 0 


DO 215 12=1,3 
DO 215 I=1,NCAUSE 
CUMSF(I2,I) = 0 
215 CONTINUE 


C COMPUTES THE ADJUSTED # OF TRIALS FROM THE ADJUSTED # OF FAILURES 
C AND COMPUTES THE SUM OF THE ADJUSTED # OF TRIALS FOR ESTIMATE COMP. 


PREL = 0. 


0 
LTRIAL = 0 


C IF ADJUSTED FAILURES ARE APPROACHING 0 THEN ADJUSTED TRIALS MUST 
U DE DEL S 


64 


DOR240 I=1,J 


IFCACIADJF,I) .LE. . 0000001) THEN 
A(IADJF,I) = . 0000001 
ENDIF 


A(IADJT,I) = A(INTR,I)/A(IADJF,I) 
TADJT = TADJT + A(IADJT,I) 


C COMPUTE YJK FROM THE ADJUSTED ; OF TRIALS AND STORE THE SUM FOR 
C ESTIMATE COMPUTATION, USE ARRAY FOR # TRIALS < 1000 AND APPROX. FOR 
C VALUES > 1000 


N1 = NINT(A(IADJT,I)) 
Mi LE. 1000) THEN 
ACIYJK, 1) = YJK(N1) 
ELSEIF(N1. GT. 1000) THEN 
X=N1 
Q= 12X 
I=X+1 
S=X+2 


AI DO) EU IO) CUDASS 


LLSE 
ENDIE 


C DETERMINE IF A PHASE BOUNDARY HAS BEEN REACHED TO BEGIN ESTIMATE 
G COMPUTATION 


USES GOTO 225 
IF(ACIPHASE, 1). NE. ACIPHASE,(1-1))) THEN 


C COMPUTE THE WOODS WEIGHTED AVERAGE ESTIMATE 


MAX = 0 
Kl < Kl + 1 


C DETERMINE THE FAILURE CAUSE WITH THE LARGEST # OF FAILURES 


DO 220 I1=1,NCAUSE 
IF(CUMSF( 1,11). GT. MAX) THEN 
MAX — CUMSF(1,11) 
[sam Z TY 
ELSE 
ENDIF 
220 CONTINUE 


C COMPUTE YJK VALUE FOR THE CURRENT PHASE ESTIMATE 


IPEOUMSE(C 1, ICOL). LE. 1000) THEN 
AHATL = YJK(CUMSF(1,1COL)) 
ELSEIF(CUMSF( 1, ICOL). GT. 1000) THEN 
X = CUMSF(1,ICOL) 
Q=12*X 
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=X+1 
S=X+2 


AHATE=(EUL+(LOG(X))+(1/(2*X))-(1/(0*T))-(1/(0*T*5))) 


ELSE 
ENDIF 
IX S=°CUMSFOI TCO Sr Curr. leo, 
IF( IX. LE. 1000) THEN 

AHATU = YJK(IX) 
ELSEIF( IX. CT: 0000) SL 

X = IX 

0=12*X 

T=X+1 

S=X+2 


AHATU-(EULH LOG(X) )+( 1/ (25X) ) 01/000 1000] U ED 


ELSE 
ENDIF 


COMPUTE CURRENT PHASE RELIABILITY ESTIMATE 
AHAT < AHATU - AHATL 
CREL = 1.0 - EXP( -AHAT) 
X = CUMSE(1,IGOL) + GUNSEU5, ICOL) 
COMPUTE AND STORE THE WOODS WEIGHTED AVERAGE ESTIMATE 


PREL = ((LTRIAL*PREL)/X) + (((X-LTRIAL)*CREL)/X) 
LTRIAL = CUMSF(1,ICOL) + CUMSF(3,ICOL) 


COMPUTE THE PHASE AND GLOBAL si FOR YJK USED IN THE EXPONENTIAL 
REGRESSION ESTIMATES ARE 


REG(2,K1) = TYJK/(I-1) 
REG(3,K1) = TPYJK/NFAPH(K1) 
TPYJK = 0.0 

ENDIF 


COMPUTE THE NUMBER OF FAILURES AND SUCCESSES FOR EACH FAILURE CAUSE 
USED IN THE WOODS WEIGHTED AVERAGE ESTIMATE 


225 ICAUSE = INT(A(IFAILC,I)+.5) 
DO 230 I1=1,NCAUSE 
CUMSF(2,I1) = CUMSF(2,I1) + INT(ACINTR,I) +.5) 
CUMSF(3,11) = CUMSF(3,11) + Ni 
230 CONTINUE 
CUMSF(1,ICAUSE) = CUMSF(1,ICAUSE) + 1 
CUMSF(2,ICAUSE) = CUMSF(2,1CAUSE) - 1 
CUMSF(3,ICAUSE) = CUMSF(3,ICAUSE) - 1 
TPYJK = TPYJK + ACIYJK,I) 
TYJK = TYJK + ACIYJK, 1) 
240 CONTINUE 
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C REPEAT COMPUTATIONS FOR THE WOODS WEIGHTED AVERAGE ESTIMATE FOR THE 
C FINAL PHASE 


245 


MAX = 0 

NN 

DO 245 11=1,NCAUSE 
IF(CUMSF(1,11). GT. MAX) THEN 
MAX = CUMSF(1,I1) 
ICOL = Il 
ELSE 
ENDIF 

CONTINUE 


IF(CUMSF( 1,ICOL). LE. 1000) THEN 
AHATL = YJK(CUMSF(1,ICOL)) 
ELSEIF(CUMSF(1,ICOL). GT. 1000) THEN 
X = CUMSF(1,ICOL) 
Q=12%X 
=X+1 
S=X+2 


ARATL=(EUL+( LOG(X) )+(1/(2*X) )-(1/(0*T))-(1/(Q*T*S))) 


ELSE 
ENDIF 
PARCS OL TOOL) +" CUMSF( 3, ICOL) 
IF(IX. LE. 1000) THEN 

AHATU = YJK(IX) 
ELSEIF(IX. GT. 1000) THEN 

X = IX 

Q=12*X 

T=X+1 

S=X+2 


AHATU=(EUL+( LOG(X))+(1/(2*X))-(1/(Q*T))-(1/(Q*T*S))) 


ELSE 
ENDIF 


AHAT = AHATU - AHATL 
kom 0 -CESPC-AHAT) 
X = CUMSF(1,ICOL) + CUMSF(3,ICOL) 


PREL = ((LTRIAL*PREL)/X) + (((X-LTRIAL)*CREL)/X) 
LTRIAL = CUMSF(1,ICOL) + CUMSF(3,ICOL) 

REG(2,K1) = TYJK/(J) 

REG(3,K1) = TPYJK/NFAPH(K1) 


POREST(CI,K) = PREL 


Č COMPUTE THE MLE ESTIMATE OF PHASE RELIABILITY USING DISCOUNTING 


EHKESIU2.K) = (TADJT = J)/TADIT 


C COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE BEGINNING WITH B HAT 
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250 


SUM = 0.0 


SUMS = 0.0 
IF (KFO |) Com) 25? 
DO 25-007 Zl 
SUM = SUM + ((I-REG(1,K))*REG(3,1)) 
SUMS = SUMS + ((I-REG(1,K))**2) 
CONTINUE 
REG(4,K) = SUM/SUMS 


C COMPUTE A HAT 


REGUSSS) REG(2,K) > (REG RECIFE) 


C COMPUTE AND STORE THE EXPONENTIAL REGRESSION ESTIMATE 


PIREST AE NOE KREGS aa 
TF(PHREST(4 JET OD PENE 
SOTO ZSR 


REG(4,K)*K))) 


252 PHRES ICS, 8) > 1,0 - PE 0 b J 
IF(PHREST(4,K). LT. 0. 0) PARES T MP0 0 
C STORE THE RUNNING SUM OF THE ESTINATES FOR THE CURRENT PHASE AND THE 
C RUNNING SUM OF THE ESTIMATES SQUARED FOR COMPUTATION OF THE MEAN AND 
C STANDARD DEVIATION OF EACH ESTIMATE FOR EACH RELIABILITY GROWTH 
C MODEL 
255 Ui (LK) ES LCI RES) 


EST(2,K) = EST(2,K) + (PHREST(1,K)**2) 
MLEWD(1,K) = MLEWD(1,K) + PHREST(2,K) 
MLEWD(2,K) = MLEWD(2,K) + (PHREST(2,K)**2) 
REGEST(1,K) = REGEST(1,K) + PHREST(4,K) 
REGEST(2,K) = REGEST(2,K) + (PHREST(4,K)**2) 


C STORE THE ACTUAL PHASE RELIABILITY 


AREL(K) = A(ISYSPR,J) 


C PRINT INTERMEDIATE OUTPUT IF REQUESTED AND THE NUMBER OF REPETITIONS 


ETS 


1000 
1010 
1020 
1022 


1025 


NOT GREATER THAN 5 


IFCLURI:NE Coon 
[PC REP, GE 8601000 


WRITE( 20,1000) REP,K 

FORNAT(T16,' REPETITION NUMBER: ',I4,' PHASE NUMBER: ',14) 
WRITE(20,1010) A(ISYSPR,J) 

FORMAT( 22X, ACTUAL COMPONENT RELIABILITY: ',F7.5) 
WRITE(20,1020) PHREST(1,K) 

FORMAT( 20X,' PREDICTED COMPONENT RELIABILITY: ',F7.5) 
WRITE(20,1022) PHREST(2,K) 

FORMAT(20X,'MLE ESTIMATE USING DISCOUNTING: ',F7.5) 
WRITE(20,1025) PHREST(3,K) 

FORMAT( 18X, MLE ESTIMATE OF PHASE RELIABILITY: ',F7. 5) 
WRITE( 20,1027) PHREST(4,K) 
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1027 FORMAT(14X, REGRESSION ESTIMATE OF PHASE RELIABILITY: ',F7.5) 
WRITE(20,1030) 
1030 FORMAT(' ',' >) 
DO 260 I=1,NCAUSE 
NRITE(20,1035)I,A(I,J) ,A((I+NCAUSE) „J) 
IOFPASTORNAT(I2X CAUSE: 13," PROSUCCESS): ',F7.6,' 4 TRIALS: ' 
CF10.0) 
260 CONTINUE 
WRITE( 20, 1036) 
1056 FORMAT(  "," ') 
WRITE(20,1040) 
1040 FORMAT(4X, FAIL 4 ,3X, FAIL CAUSE' ,3X, '4ł TRIALS' ,3X,'ADJ # FAIL' ,3 
SRDA 7 TRIALS' 7X, YJK') 
POM: I=1,J 
a OC CO 
s] 
1050 FORMAT(4X,I3,8X,F3.0,7X,F8.0,4X,F8.6,4X,F12.0,3X,Fli.4) 
270 CONTINUE 
WRITE(20,1060) 
1060 FORMAT(' ',///) 


3 


300 CONTINUE 


U FRINI EACH OF THE 3 ESTIMATES TO THEIR APPROPRIATE OUTPUT FILE 
te lh REQUESTED 


IF( IOPT1.NE. 1) GOTO 401 
400 WRITE(40,2000) (PHREST(1,I), I=1,NPHASE) 
401 IF(IOPT2.NE. 1) GOTO 402 

WRITE(50,2000) (PHREST(2,I), I=1,NPHASE) 
402 IF(IOPT3.NE. 1) GOTO 403 

WRITE(60,2000) (PHREST(3,I), I=1,NPHASE) 
403 IF(IOPT4.NE.1) GOTO 500 

WRITE(70,2000) (PHREST(4,I), I=1,NPHASE) 
2000 FORMAT(' ',30(F7.6:1X)) 


500 CONTINUE 


C UPON COMPLETION OF ALL REPETITIONS, COMPUTE THE MEAN AND STANDARD 
C DEVIATION OF EACH ESTIMATE FOR EACH PHASE SKIPPING COMPUTATIONS IF 
C ONLY ONE REPETITION IS REQUIRED 


IE (NREPREES1) GOTO 601 


DO 600 I=1,NPHASE 

EST(3,1) — EST(1,1)/XNREP 

MLEWD(3,1) = MLEWD(1,I)/XNREP 

MLESP(3,1) = MLESP(1,1)/XNREP 

REGEST(3,1) — REGEST(1,1)/XNREP 

EST(4,I) = SQRT((EST(2,1)-(XNREP*(EST(3,1)**2)))/(XNREP-1)) 

MLEWD(4,I) = SQRT((MLEWD(2,1)-(XNREP*(MLEWD(3,I)?*2)))/(XNREP-1)) 

MLESP(4,I) = SORT((MLESP(2,I)-(XNREP*(MLESP(3,1)**2)))/(XNREP-1)) 

REGEST( 4, I )=SQRT((REGEST( 2, I) -( XNREP*( REGEST( 3, 1)**2)) )/(XNREP-1)) 
600 CONTINUE 


C PRINT THE FINAL OUTPUT TABLE TO Å FILE 
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601 WRITE(30,3000) 
3000 FORMAT('0',T47,'DISCRETE RELIABILITY GROWTH SIMULATION' ) 
WRITE(30,3010) 
3010 FORMAT('-',T54,'MODEL PARAMETER SUMMARY' ) 
WRITE(30,3020) NCAUSE 
3020 FORMAT('0',T47,'NUMBER OF POSSIBLE FAILURE CAUSES ',I4) 
IF (FRELOP .EQ. 1) GOTO 4000 
WRITE(30,3030) 
3030 FORMAT('0',T38,'CAUSE NUMBER',T64,'SINGLE TRIAL PR( SUCCESS ) FOR 
CPHASE 1') 
DO 3050 M=1,NCAUSE 
WRITE(30,3040) M,A(M,1) 
3040 FORMAT(' ',T43,I2,T79,F8.6) 
3050 CONTINUE 
WRITE(30,3060) FRIMP 
3060 FORMAT( 0 ,T37, FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 
C',F8.6) 
5000 WRITE(30,3080) NPHASE 
3080 FORMAT('-',T48,'NUMBER OF PHASES IN THE SIMULATION ',I2) 


WRITE(30,3090) 
3090 FORMAT( "0" ,T42, PHASE NUMBER" ,T59, NUMBER OF FAILURES IN THE FIRST 
C PHASE") 


DO 3110 M=1,NPHASE 
WRITE(30,3100) M,NFAPH(M) 
3100NFORNAT( FS TN 
3110 CONTINUE 
WRITE(30,3120) NFAIL 
3120 FORMAT('0',T51,'TOTAL NUMBER OF FAILURES ',14) 
IF(DISOPT.EQ.2) GO TO 3160 


WRITE(30,3130) 
3130 FORMAT('-',T38, DISCOUNTING PERFORMED USING THE CONSTANT FRACTION 
CMETHOD' ) 


WRITE(30,3140) R 
3140 FORMAT('0' ,T44,'FRACTION EACH FAILURE IS DISCOUNTED ',F8.6) 
WRITE(30,3150) N 
3150 FORMAT(' ',T33,'NUMBER OF TRIALS AFTER A FAILURE BEFORE A DISCOUNT 
CIS APPLIED TII 
GO TO 3190 
3160 WRITE(30,3170) 
3170 FORMAT(' -* ,T44,'DISCOUNTING PERFORMED USING THE LLOYD METHOD") 
WRITE(30,3180) GAMA 
3180 FORMAT('0',T39, PERCENT C.I. ( USED AS DISCOUNT FRACTION ) ',F8.6 
C) 
WRITE(30,3185) LDI 
3185 FORMAT('0' ,T50,' LLOYD DISCOUNT INTERVAL: ',I3) 
3190 WRITE(30,3200) DSEEDI 
3200 FORMAT('-',T46,'RANDOM NUMBER SEED USED ',F15.2) 
WRITE(30,3210) NREP 
3210 FORMAT('0' ,T37, NUMBER OF REPETITIONS OF THE SIMULATION PERFORMED 
CZAI 
WRITE(30,3220) 
3220 FORMAT('1' ,T61,' ESTIMATOR: ') 
WRITE(30,3230) 
3230 FORMAT('0',T48,'SINGLE PHASE MLE WITHOUT DISCOUNTING') 
WRITE(30,3240) 
3240 FORMAT( -',T60, 'MEAN' ,T83, ESTIMATE' ,T109, '95 %') 
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3250 


WRITE(30,3250) 
FORMAT(' ',T12,' PHASE NUMBER',T29, 'ACIUAL RELIABILITY',T52, ' PREDIG 


E RELIABILITY' LBS ' STANDARD DEVIATION' JO ' CONFIDENCE INTERVA 


C COMPUTE C.I. FOR SINGLE PHASE MLE 


3260 


2270 


3280 


DO 3270 M=1,NPHASE 

CI = (1.96*MLESP(4,M))/SORT(XNREP) 

CIU = MLESP(3,M) + CI 

CIL = MLESP(3,M) - CI 

WRITE(30, 3260) M,AREL(M) ,MLESP(3,M) ,MLESP(4,M), CIL,CIU 

FORMAT( ， 0',T17, 12, T34,F8.6,T58,F8.6,T82,F9.6,T99,' ( SEDMO. "F 


010, )) 


CONTINUE 

PNL 010553220) 

WRITE(30,3280) 

FORMAT('0' ,T42,'MAX LIKELIHOOD ESTIMATE USING DISCOUNTED FAILURES' 


WRITE( 30,3240) 
WVRITE( 30,3250) 


GC COMPUTE C.J. FOR MLE WITH DISCOUNTING 


5290 


3300 


DO 3290 M=1,NPHASE 
= (1. 96*MLEWD( 4 ,M) ) /SQRT( XNREP) 
CIU = MLEWD(3,M) + CI 
CIL = MLEWD(3,M) - CI 
WRITE(30,3260) M,AREL(M),MLEWD(3,M),MLEWD(4,M),CIL,CIU 
CONTINUE 
WRITE(30,3220) 
WRITE(30,3300) 
FORMAT('0',T38,'WEIGHTED AVERAGE ESTIMATE USING FAILURE DISCOUNTIN 


CG') 


WRITE( 30,3240) 
WRITE( 30,3250) 


C COMPUTE C.I. FOR WOODS WEIGHTED AVERAGE ESTIMATES 


SOLO 


3320 


DO 3310 M=1,NPHASE 

I = (1.96*EST(4,M))/SORT(XNREP) 

CIU = EST(3,M) + CI 

CUL = EST(3.M) - CI 

WRITE(30,3260) M,AREL(M),EST(3,M),EST(4,M),CIL,CIU 
CONTINUE 


WRITE(30,3220) 
WRITE(30,3320) 
FORMAT('0',T43,'REGRESSION ESTIMATE USING DISCOUNTED FAILURES') 
WRITE(30,3240) 
WRITE(30,3250) 


C COMPUTE C.J. FOR EXPONENTIAL REGRESSION ESTIMATES 


DO 3330 M=1,NPHASE 
CI = (1. 96*REGEST(4,M))/SORT(XNREP) 
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CIU = REGEST(3,M) + CI 
CIL — REGEST( SU 5 0] 
WRITE(30,3260) M,AREL(M),REGEST(3,M),REGEST(4,M),CIL,CIU 
3330 CONTINUE 
WRITE(30,3340) 
3340 FORMAT('1',T59, RECAPITULATION '//) 
WRITE(30,3350) 
3350 FORMAT('-',T3,'PHASE' ,T11,'ACTUAL' ,T28,'MEAN' ,T38 
C,T63, EST" ,T78, MEAN „,T68, EST ,T103, MEAN ER 
WRITE(30,3360) 
3360 FORMAT(' ',Tll 
C7, "PHASE" ,T88, 
WRITE( 30,3370) 
3370 FORMAT(' ',T28, 'AVG',T35,' DEVIATION' ,T53,'W/D',T60, DEVIATION: ,T78 
C, MLĘ' ,T85, DEVIATION ,T103, EST TIN DET 
WRITE(30,3375) 
3375 FORMAT“ NE 
DO 650 I=1,NPHASE 
WRITE(30,3380) I ,AREL(TLESTO, DESTO.) MØ NN NR 
CMLESP(3,I),MLESP(4,I),REGEST(3,I),REGEST(4,I) 
3380 FORMAT( 0" ,T4,13,711,F7.6,726,F7.6,T36 E7.6 151 FAN ON 
CE7.6 T86 ,P7.ó 1101,17.6 MERITE), 
650 CONTINUE 
GO TO 6000 


, EST! ,T53, 'MEAN' 
EST') 
'RELIAB’ ,128,'WGT' ,T38,'SED' ,T53, MES ne 


'STD',T103,'REG',T113,'STD') 


4000 WRITE(30,4010) 

4010 FORMAT( 1X,//,T50, FIXED PHASE RELIABILITY OPTION') 
WRITE( 30,4020) 

4020 FORMAT('-',T38, PHASE NUMBER' ,T78,' ACTUAL RELIABILITY ') 
DO 4030 M=1,NPHASE 

WRITE( 30,4040) M,AREL(M) 

4040 FORMAT( "0" ,T41,12,783,F8. 6) 

4030 CONTINUE 
GO TO 5000 

6000 CONTINUE 


STOP 
END 


APPENDIX C. GRAPHICAL RESULTS 


This appendix contains the results of many of the various runs done with the sim- 
ulation. These results appear in tabular format in the output file but have been reduced 
to graphical form for ease of understanding. Below each graph of the estimates produced 
by the growth models are graphs of the standard deviations of each estimate bv phase. 


This appendix is ordered by reliability growth pattern with Pattern I first and the re- 


mainder following sequentially. 
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Pattern I, F = .25, 1 = 1 


mo 


1.0 


RELIABILITY 
0.6 0.8 


0.4 


0.2 





on 
o 


STANDARD DFVIATION 


0.1 


Figure 29. 


= 
A = 
3 = —+— ACTUAL RELIABILITY 
= = —[— SINGLE PHASE MLE 
—V— EXP REG 
—+— MLEFD 
1 
PHASE 
Di 
Z 4 6 8 10 
PHASE 


Pattern I, F = .25, 1 = 3 
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Figure 30. Pattern I, F = .25,1=6 
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Figure 31. Pattern I, F = .50, 1 = 3 
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Pattern I, F = .50, 1 = 6 
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34. Pattern I, F = .75,1 = 6 
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Pattern I, Lloyd, CI = .8 
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Figure 36. Pattern I, Lloyd, CI = .9 
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Figure 38. Pattern II, F = .25,1= 3 
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Pattern II, F = .50, I = 3 
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Figure 42. Pattern II, F = .75, 1 = 3 
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Figure 44. Pattern II, Lloyd, CI = .3 
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Figure 46. Pattern III, No Discounting 
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Figure 48. 


Pattern III, F = .25,I 
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Pattern III, F = .25,1=6 
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Figure 54. 





Pattern III, Lloyd, CI = .8 
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Figure 55.  Pattern III, Lloyd, CI < .9 
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Figure 56. Pattern IV, No Discounting 


103 


RELIABILITY 


—*— ACTUAL RELIABILITY 
-0— MLE SINGLE PHASE 





; E RE 
> —+— MLEFD 
= | 
2 4 6 8 10 
PHASE 


STANDARD DEVIATION 





Figure 37. Pattern IV, F = .10 1 = 10 
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Figure 59. Pattern IV, F = .25,1 = 3 


Ua 


106 


RELIABILITY 


—*— ACTUAL RELIABILITY 
—0— MLE SINGLE PHASE 





h —V- EXP REG 
o —+= MLEFD 
O 
2 4 6 8 10 
PHASE 


STANDARD DEVIATION 





Figure 60.  Pattern IV, F < .50, I < 60 


107 


= n o on jā ED 


RELIABILITY 


—*— ACTUAL RELIABILITY 
—U- MLE SINGLE PHASE 





s -V- EXP REG 
o —+— MLEFD 
o | | 
2 4 6 8 10 
PHASE 


STANDARD DEVIATION 





PHASE 


Figure 61. Pattern IV, Lloyd, CI = .8 
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Figure 62. Pattern IV, Lloyd, CI = .9 
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Figure 63. Pattern IV, Lloyd, CI = .99 
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Pattern V, No Discounting 
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Figure 65. Pattern V, F = .25,1= 1 
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Figure 66. Pattern VY, F = 25,1 = 3 
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Pattern V, F = .50,1 = 3 
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Pattern V, F = .75,I = 3 
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Figure 72. 


Pattern V, Lloyd, CI = .3 
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Pattern V, Lloyd, CI = .9 
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Figure 75. 


Pattern VI, F = .25,I 
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Figure 76. Pattern VI, F = .25, 1 = 3 
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Figure 77. Pattern VI, F = .25,I = 6 
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Figure 79. Pattern VI, F = .30, [I = 6 
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Figure 82. 


Pattern VI, Lloyd, CI = .8 
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Figure 83. Pattern VI, Lloyd, CI = .9 
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Figure 84. Pattern VII, No Discounting 
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Figure 37. Pattern VII, F = .25,1 = 6 
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Figure 88. Pattern VII, F = .50, I = 3 
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Figure 89. Pattern VII, F = .50,I = 6 
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Figure 90. Pattern VII, F = .75,1 = 3 
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Pattern VIIL, F = .75, 1 = 6 
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Figure 92. Pattern VII, Lloyd, CI = .8 
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Figure 93. Pattern VII, Lloyd, CI = .9 
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Figure 94. Pattern VIII, No Discounting 
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Pattern VIII, F = .25, 1 = 3 
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Figure 97. Pattern VIII, F = .25,1 = 6 
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Figure 98. Pattern VIII, F = .50,1 = 3 
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Figure 100. Pattern VIII, F = .75,1= 3 
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Figure 101. Pattern VIII, F = .75,1 = 6 
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Figure 102. Pattern VIII, Lloyd, CI = .8 
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